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Abstract 


The  Cold  War  may  be  over  but  the  force  modernization  war  has  just  begun.  If  the 
Air  Force  is  going  to  meet  its  goals  established  in  Joint  Vision  2001,  ways  must  be  found 
to  modernize  aging  weapon  systems  in  the  face  of  decreasing  defense  budgets  and 
personnel  resources.  In  a  hurried  effort  to  realize  the  savings  necessary  to  continue  with 
modernization,  Air  Force  leaders  have  embraced  the  philosophy  of  outsourcing. 
Proponents  claimed  that  outsourcing  would  reap  vast  savings,  but  recent  General 
Accounting  Office  (GAO)  audits  confirm  that  the  savings  from  outsourcing  have  not 
been  as  high  as  originally  projected.  These  recent  GAO  reports  suggest  that  problems 
with  the  way  that  cost  estimating  is  done  for  outsourcing  competition  studies  may 
overstate  the  economic  advantages  of  this  alternative  to  organic  support  (GAO  Report, 
GAO/NSI  AD-97-86, 1997). 

The  question  Air  Force  leaders  want  to  know  is;  can  we  determine  whether  or  not 
outsourcing  is  a  better  value  than  maintaining  a  capability  in-house?  A  unique  situation 
has  arisen  at  the  Sacramento  Air  Logistics  Center  (ALC)  where  control  of  depot-level 
maintenance  for  the  F-l  17  was  transferred  from  government  to  contractor  control.  Now 
that  control  of  the  depot-level  maintenance  of  the  F-l  17  has  been  outsourced,  we  wanted 
to  find  out  if  outsourcing  was  a  better  value  than  the  previous  government  controlled 
support.  This  study  used  archival  cost  data  and  mission  capability  factors,  top  level 
management  interviews,  and  a  customer  satisfaction  questionnaire  to  examine  the  relative 
value  of  outsourcing  depot  support  of  the  F-l  17.  Triangulating  these  three  value 
elements  helped  us  determine  if  outsourcing  cost  less,  improved  readiness,  and  increased 


customer  satisfaction  when  compared  to  government  controlled  depot-level  maintenance. 
Our  analysis  found  that  depot  maintenance  of  the  F-117  since  transferred  to  contractor 
control  is  a  better  value  to  the  Air  Force.  Technical  performance  has  been  equal  or  better 
and  savings  of  approximately  $25  million  is  projected  for  the  first  year. 


BEFORE  AND  AFTER:  IS  OUTSOURCING  A  BETTER  VALUE  THAN 


ORGANIC  SUPPORT? 

A  CASE  STUDY  COMPARING  ORGANIC  VS.  CONTRACTOR  CONTROLLED 
DEPOT-LEVEL  MAINTENANCE  OF  THE  F-l  17 

I.  Introduction 


Background 

With  the  end  of  the  Cold  War,  the  role  of  the  Department  of  Defense  (DoD)  has 
radically  changed.  Since  we  are  no  longer  challenged  by  another  superpower  and  the 
main  threat  to  the  United  States  comes  in  the  form  of  regional  conflicts,  the  role  and 
mission  of  the  American  military  are  less  clear.  As  a  result,  support  for  large  defense 
budgets  has  waned.  Since  the  mid-1980s  there  has  been  an  almost  continuous  reduction 
in  defense  spending.  We  have  witnessed  the  withdrawal  of  troops  from  overseas, 
reductions  in  civilian  and  military  personnel,  base  closures,  and  reductions  of  major 
weapon  system  acquisitions. 

In  1993,  Vice  President  Gore  announced  the  National  Performance  Review  (NPR) 
along  with  his  plan  for  the  reinvention  of  Government.  With  NPR,  came  an  increased 
emphasis  on  improving  fundamental  internal  processes.  In  its  most  basic  form,  NPR 
builds  on  four  key  principles:  (1)  cutting  red  tape;  (2)  putting  customers  first; 

(3)  empowering  employees  to  get  results;  and  (4)  cutting  back  to  basics:  producing  better 
government  for  less.  One  strategy  being  explored  to  achieve  NPR’s  efficiency  goals  is  to 
encourage  federal  organizations  to  explore  the  viability  of  having  private  companies 
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perform  all  non-inherently  governmental  work.  As  the  largest  spender  of  discretionary 
funds,  the  DoD  is  one  federal  agency  that  can’t  afford  to  do  things  in  its  usual  way, 
especially  when  private  firms  may  be  able  to  perform  DoD’s  non-core  functions  better, 
cheaper,  and  faster.  “Functions  such  as  command,  deployment,  or  rotation  of  troops 
cannot  be  contracted,  of  course.  But  data  processing,  billing,  payroll,  and  the  like 
certainly  can”  (Gore,  1993:  58).  The  following  quote  summarizes  the  overall  philosophy 
of  the  NPR: 

We  can  no  longer  afford  to  pay  more  -  and  get  less  from  -  our 
government.  The  answer  for  every  problem  cannot  always  be  another 
program  or  more  money.  It  is  time  to  radically  change  the  way  the 
government  operates  -  to  shift  from  top-down  bureaucracy  to 
entrepreneurial  government  that  empowers  citizens  and  communities 
to  change  the  country  from  the  bottom  up.  We  must  reward  the  people 
and  ideas  that  work  and  get  rid  of  those  that  don’t.  (Gore,  1993:  58) 

A  dwindling  Air  Force  budget,  increased  operations  tempo,  and  continuing 

manpower  reductions  have  presented  Air  Force  leaders  with  retention  problems  and  the 

challenge  of  doing  more  with  only  62  percent  of  the  budget  and  65  percent  of  the 

manpower  often  years  ago.  In  1989  the  Air  Force  manpower  level  was  570,880  with  a 

budget  of  $121  billion.  In  1998  manpower  was  reduced  to  371,577  and  available  funding 

dropped  to  $76  billion  ("The  Air  Force  in  Facts  and  Figures",  Air  Force  Magazine,  VoL 

48. 1998:  36  &  46).  Air  Force  leaders  began  eliminating  unnecessary  infrastructure 

(closing  bases),  and  recouping  the  associated  infrastructure  costs.  Opponents  of  these 

closures  created  a  debate  in  Congress  that  resulted  in  reduced  effectiveness  of  the  Air 

Force’s  primary  means  of  saving  money.  Political  lines  were  drawn,  and  Congressmen 

fought  to  see  who  would  control  what  some  would  call  "government  job  welfare" 

(Kitfield,  1998:  28-31).  The  battles  and  long  standing  debate  resulted  because  the 
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management  plan  to  quickly  close  McClellan  Air  Force  Base  was  not  implemented 
decisively.  This  resulted  in  a  compromise  to  privatize  in-place  the  work  of  the 
Sacramento  ALC  (Kitfield,  1998: 28-31). 

In  spite  of  these  difficulties,  the  Air  Force  has  stepped  up  its  pursuit  of 
outsourcing  any  function  that  is  not  an  inherently  governmental  function:  "Additional 
outsourcing  studies  involving  more  than  100,000  positions  will  be  started  over  the  next 
six  years"  (GAO  Report,  GAO/NSI AD-97-86, 2-7, 1997).  The  primary  areas  considered 
for  outsourcing  were  base  support  functions  and  depot-level  maintenance.  Initially, 
proponents  claimed  that  outsourcing  could  save  the  Department  of  Defense  billions  of 
dollars.  However,  several  recent  General  Accounting  Office  (GAO)  reports  regarding 
cost  estimating  practices  and  other  commercial  sector  warning  flags  have  caused  us  to  re¬ 
evaluate  the  projected  savings  generated  from  outsourcing.  This  is  a  key  factor  in  Future 
Years  Defense  Budgets  because  current  Air  Force  weapons  programs  have  used  these 
projected  savings  in  their  program  budget  estimates  (Peters,  1999:  21).  The  question  now 
is:  Is  outsourcing  a  better  value  than  organic  support ?  This  question  demonstrates  the 
need  to  do  more  research  on  the  costs  and  benefits  of  outsourcing  before  proceeding 
further. 

In  her  thesis  entitled  "Outsourcing:  An  Historical  Review  For  The  Projection  Of 
Future  Savings,"  Captain  Leslee  J.  Saleck  reviews  both  historical  and  current  trends  in 
outsourcing  along  with  the  conditions  that  contribute  to  a  successful  outsourcing 
relationship  with  the  contractor.  She  warns  that,  "The  DoD  should  be  cautious  about 
wholeheartedly  embracing  the  Commission  on  Roles  and  Missions  (CORM)  and  Defense 
Science  Boards  (DSB)  recommendations  to  outsource  essentially  every  non-war-fighting 
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activity.  Though  there  is  potential  for  savings  through  outsourcing,  it  is  highly  dependent 
on  the  existence  of  multiple  conditions  (Saleck,  1998:  74)."  The  conditions  that  represent 
the  most  feasible  environment  for  outsourcing  are:  (1)  the  work  to  be  done  is  clearly 
specified;  (2)  several  potential  producers  are  available,  and  a  competitive  climate  either 
exists  or  can  be  created  and  sustained;  (3)  the  government  is  able  to  monitor  the 
contractor’s  performance;  and  (4)  appropriate  terms  are  included  in  the  contract 
document  and  enforced  (Savas,  1987: 109).  This  research  examines  the  differences  in 
cost  and  performance  of  government  controlled  depot  level  maintenance  and  contractor 
controlled  depot  level  maintenance,  and  suggests  a  method  of  measuring  the  value  gained 
or  lost  from  outsourcing. 

Problem  Statement 

The  primary  goal  of  this  research  was  to  examine,  through  triangulation  of 
archival  cost  and  performance  data,  expert  interviews,  and  a  customer  satisfaction 
questionnaire,  whether  or  not  the  Air  Force  was  receiving  a  better  value  by  outsourcing 
total  responsibility  for  the  depot-level  maintenance  of  the  F-l  17.  Part  of  this  primary 
goal  is  to  design  a  questionnaire  that  measures  the  customer’s  perception  of  the  value 
received  from  outsourced  depot-level  maintenance  of  the  F-l  17.  Throughout  this 
research  the  value  determination  measured  is  the  comparison  of  government  controlled 
depot-level  maintenance  versus  contractor  controlled  depot-level  maintenance.  The 
results  of  this  study  are  representative  of  the  F-l  17  only  and  do  not  reflect  the  value  of 
outsourcing  to  the  Air  Force  as  a  whole.  A  secondary  goal  was  to  develop  research 
procedures  that  may  be  used  in  future  multiple  case  studies.  These  procedures  attempt  to 
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highlight  the  cost  and  performance  of  organic  support  compared  to  the  cost  and 
performance  of  outsourcing  in  an  effort  to  provide  a  method  of  quantifying  the  overall 
value  attained  from  outsourcing.  It  is  our  hope  that  this  triangulated  comparative  method 
can  be  used  to  evaluate  whether  or  not  the  Air  Force  is  receiving  a  better  overall  value 
from  outsourcing. 

Criticisms  of  government  estimating  practices  during  outsourcing  competitions 
arose  from  industry  and  initiated  an  effort  to  equalize  estimating  practices.  Recently,  the 
GAO  published  similar  criticisms  and  has  confirmed  that  the  savings  derived  from 
outsourcing  are  not  as  great  as  originally  projected  (GAO  Report,  GAO/NSI AD-98-48, 
1997).  A  recent  decision  to  outsource  the  responsibility  for  depot  level  maintenance  of 
the  F-l  17  aircraft  has  made  it  possible  to  study  a  weapon  system  that  transitioned  from 
government  control  of  sustainment  functions  to  complete  contractor  control.  By 
gathering  actual  cost  and  performance  data  and  comparing  it  to  the  customer  satisfaction 
questionnaire,  we  were  able  to  determine  if  the  data  collected  reflected  the  opinions  of 
the  customers.  Further,  by  triangulating  the  cost  and  performance  data  and  the  opinion 
surveys  we  hoped  to  gain  some  insights  into  how  best  to  shape  value  measures  in  order  to 
promote  best  value  decision-making  in  these  critical  support  areas. 

Investigative  Questions 

The  investigative  questions  are  designed  to  focus  the  research  on  an  examination 
of  contractor  performance  data,  cost  benefits,  and  perceived  customer  satisfaction. 

1 .  Are  the  contract  performance  metrics  good  indicators  of  better  value ? 

2.  Has  the  decision  to  outsource  F-l  17  depot-level  maintenance  provided  a  better  value 
to  the  Air  Force? 
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3.  If  outsourcing  F-l  17  depot  maintenance  is  an  equal  or  better  value,  do  we  still  want  to 
give  up  the  potential  core  capability? 

Research  Focus 

The  factors  studied  here  are  the  differences  in  cost  and  performance  resulting 
from  the  experience  of  having  both  organic  and  contractor  controlled  depot-level 
maintenance.  The  goal  is  to  assess  F-l  17  outsourcing  to  determine  if  it  is  a  better  value 
to  the  Air  Force  than  organic  support.  The  plan  is  to  measure  the  actual  differences  in 
performance  factors  and  then  to  triangulate  the  archival  and  secondary  data,  expert 
interviews,  and  customer  satisfaction  questionnaire  results  to  increase  the  internal  and 
external  validity  of  this  single  case  study  (Yin,  1989).  The  focus  then  is  to  gather 
pertinent  cost  and  other  associated  value  factors  through  archival  data  analysis  and 
compare  them  with  the  level  of  customer  perceived  value  gathered  from  the  customer 
satisfaction  questionnaires.  Once  the  data  are  analyzed,  and  shown  to  be  either  valid  or 
invalid  measures  of  the  value  of  outsourcing,  the  researchers  will  make  recommendations 
for  further  use  of  this  information. 

Assumptions 

If  the  results  of  the  archival  data  analysis  and  customer  satisfaction  questionnaires 
are  consistent  with  our  proposition  that  outsourcing  is  a  better  value  than  organic 
maintenance,  it  means  that  the  decision  to  outsource  was  appropriate  for  the  F-l  17 
aircraft.  The  results  we  obtain  are  reliable  only  for  this  depot  and  this  particular  situation. 
If  the  results  are  not  consistent  with  our  proposition,  or  if  the  results  of  the  questionnaire 
conflict  with  the  archival  data  analysis,  additional  research  may  be  required.  A  finding 
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inconsistent  with  our  proposition  would  suggest  that  future  decisions  to  outsource 
additional  key  government  functions  should  be  thoroughly  researched.  The  most 
important  aspect  of  this  research  is  to  determine  if  outsourcing  the  F-l  17  provided  the 
best  overall  value  to  the  government  and  to  build  a  framework  for  future  decision-makers 
to  apply  when  confronted  with  a  similar  outsourcing  decision. 

Limitations 

This  study  does  not  analyze  the  political  or  social  confounds  impacting  decisions 
to  outsource.  Several  comments  have  already  been  made  about  the  inaccuracy  of  the 
methodology  used  to  estimate  costs  during  outsourcing  competitions.  The  General 
Accounting  Office  (GAO)  and  the  Office  of  Management  and  Budget  (OMB)  are 
currently  researching  how  government  costs  are  estimated  for  outsourcing  competitions 
in  an  effort  to  solve  the  problem  (GAO  Report,  GAO/NSIAD-97-86,  1997).  Because 
these  two  agencies  are  pursuing  a  solution  to  the  estimating  problem,  this  research  does 
not  address  that  area.  The  researchers  acknowledge  that  there  are  many  difficulties  with 
current  cost  estimation  practices,  and  that  their  solution  is  beyond  the  scope  of  this 
research. 

The  outsourcing  of  the  depot-level  maintenance  of  the  F-l  17  is  a  unique  situation. 
There  was  no  competition  between  the  private  and  public  sector  because  the  base  was 
authorized  for  closure;  and  although  there  was  a  most  efficient  organization  cost 
estimated  completed,  it  was  not  used  as  a  baseline  for  cost  comparison  purposes  because 
of  a  lack  of  confidence  in  its  accuracy. 
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Definitions 


Key  terms  and  definitions  necessary  to  understand  the  major  aspects  of  this 
research  are  briefly  summarized  below.  Complete  definitions  of  these  terms  are  found  in 
Appendix  A. 

Outsourcing  -  is  the  use  of  federal  funds  to  pay  a  private  company  to  do  defense 
work  or  provide  a  service  for  a  defense  activity  (GAO  Report,  GAO/NSAID-97-86, 

1997: 2). 

Privatization  -  the  complete  transfer  of  ownership  and  management  of  a  function 
to  the  private  sector,  but  DOD  pays  for  the  services  associated  with  the  function  (GAO 
Report,  GAO/NSAID-97-86, 1997:  2). 

Better  Value  -  a  construct,  in  this  framework,  that  is  measured  by  whether  or  not 
there  are  actual  improvements  in  cost,  mission  capable  ratings  (MCR's),  quality, 
timeliness,  and  reliability  of  depot  repairs. 

Modernization  -  a  key  goal  of  the  Air  Force  leadership,  outlined  in  Joint  Vision 
2001,  to  replace  or  upgrade  the  aging  Air  Force  weapon  systems  (Defense  Issues,  1996: 
1&2). 

Inherently  Governmental  Function  -  "a  function  that  is  so  intimately  related  to  the 
public  interest  as  to  mandate  performance  by  Government  employees"  (GAO  Report, 
GAO/NSAID-97-86, 1997:  2). 

Total  Non  Mission  Capable  Supply  Rate  -  (TNMCS)  is  based  upon  the  percentage 
of  aircraft  not  mission  capable  due  to  supply.  It  is  calculated  monthly  to  establish  a  12- 
month  moving  average  (F-l  17  TSPR  Performance  Metrics,  1998). 
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MICAP  Delivery  -  MICAP  response  time  will  be  calculated  monthly  to  establish  a 
twelve  month  moving  average.  This  metric  is  based  upon  the  time  taken  by  LMSW  to 
deliver  parts  necessary  to  perform  the  mission  to  HAFB  (F- 117  TSPR  Performance 
Metrics,  1998). 

RSP  Kit  Fill  Rate  -  RSP  Kit  Fill  Rates  will  be  calculated  monthly  to  establish  a 
12-month  moving  average.  This  metric  is  based  upon  the  total  number  of  pieces  on  hand 
in  both  A  and  B  kits  divided  by  the  total  number  of  pieces  authorized  (F- 117  TSPR 
Performance  Metrics,  1998). 

Depot  Quality  -  is  calculated  by  individual  aircraft  using  a  weighted  point  system 
that  is  based  upon  the  number  of  major  and  minor  discrepancies  found  after  the 
contractor  has  requested  DCMC  sign-off  for  closure  (F-l  17  TSPR  Performance  Metrics, 
1998). 

Depot  Delivery  -  The  depot  delivery  metric  is  calculated  for  each  aircraft  and  is 
based  upon  the  total  number  of  days  each  aircraft  fails  to  deliver  on  time  (F-l  1 7  TSPR 
Performance  Metrics,  1998). 

Delinquent  Efficiency  Reports  (DRs)  -  Delinquency  tracking  is  performed 
monthly  to  establish  a  12-month  moving  average.  This  metric  is  based  on  the  average 
number  of  delinquent  DRs  (F-l  17  TSPR  Performance  Metrics,  1998). 

Chapter  Summary 

Although  outsourcing  has  been  practiced  for  many  years,  the  Air  Force’s  loss  of 
base  closures  as  its  primary  means  of  saving  force  modernization  dollars  increased  the 
emphasis  on  the  use  of  outsourcing.  Originally,  proponents  claimed  that  millions  of 
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dollars  could  be  saved  by  outsourcing;  however,  recent  GAO  audits  suggest  the  savings 
are  less  than  proposed  (GAO  Report,  GAO/GGD  90-58, 1990).  Air  Force  leaders  are 
now  asking  researchers  if  they  can  measure  whether  outsourcing  is  a  better  value  than 
maintaining  a  capability  in-house.  This  thesis  uses  the  triangulation  of  archival  cost  and 
performance  data,  expert  interviews  with  those  involved  in  the  decision  to  outsource,  and 
a  customer  satisfaction  questionnaire  to  determine  the  value  of  outsourcing  the  depot- 
level  maintenance  of  the  F-l  17  to  the  DoD. 
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II.  The  Progression  of  Outsourcing 


Chapter  Overview 

This  chapter  briefly  discusses  the  history  of  outsourcing  within  private  industry 
and  the  Department  of  Defense,  and  then  highlights  some  of  the  most  recent  outsourcing 
developments.  The  chapter  continues  with  a  review  of  make  or  buy  decisions.  Then  it 
discusses  the  reasons  why  outsourcing  has  become  the  chosen  method  of  cost  savings. 
Next,  it  looks  at  regulatory  and  policy  guidance.  Then  it  examines  some  of  the  recent 
problems  and  developments  in  how  the  Air  Force  plans  to  use  outsourcing  to  save  the 
money  needed  to  modernize  its  force,  and  methods  for  determining  its  value.  Finally,  it 
addresses  the  decision  to  place  F-l  17  depot-level  maintenance  under  the  direct  control  of 
the  contractor. 

History  of  Outsourcing 

The  move  towards  outsourcing  is  not  a  recent  phenomena.  For  over  40  years  the 

government  has  looked  for  ways  to  optimize  its  budgets. 

Since  1955,  federal  agencies  have  been  encouraged  to  obtain  goods 
and  services  from  the  private  sector  through  outsourcing.  In  1966,  the 
Office  of  Management  and  Budget  (OMB)  issued  circular  A-76,  which 
established  the  federal  policy  for  the  government’s  performance  of 
commercial  activities.  In  a  1983  supplemental  handbook,  OMB 
established  procedures  for  determining  whether  commercial  activities 
should  be  outsourced.  (GAO  Report,  GAO/NSIAD  97-86, 15,  1997) 

The  end  of  the  Cold  War  brought  sweeping  changes  throughout  the 

Department  of  Defense  including  decreasing  budgets,  manpower,  and  resources.  To  deal 

with  these  changes  "the  Department  of  Defense  must  meet  three  major  challenges  defined 
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in  Joint  Vision  2001 :  readiness,  quality  of  life,  and  weapons  modernization"  (Armed 
Forces  Information  Service:  2, 1996).  Sound  business  judgement  must  be  employed 
when  evaluating  the  risks  involved  in  making  the  determination  to  outsource. 


To  Make  or  Not  to  Make 

In  his  article  "A  Guide  to  Logistics  Outsourcing,"  author  Robert  Bowman 

discusses  the  issues  of  outsourcing  and  warns: 

Outsourcing  isn’t  for  everyone.  It  isn’t  for  those  for  whom  logistics 
is  a  core  competency  or  one  that  is  essential  to  winning  and  retaining 
customers.  Companies  looking  to  make  that  initial  determination  must 
ask  themselves  a  series  of  key  questions:  (1)  What  are  my  current  strategic 
objectives  regarding  services  and  costs;  (2)  Which  activities  should  be 
“owned,”  and  which  should  be  outsourced;  and  (3)  Should  “owned” 
resources  be  reconfigured?  If  so  how?  (Bowman,  1997:  34-36) 

The  basic  reasons  for  deciding  whether  to  make  or  buy  should  not  be  based  on 

cost  alone.  Many  factors  need  to  be  considered.  “You  concentrate  on  your  core 

competencies,  and  outsource  what  other  companies  can  do  better  than  you”  (Lear-Olimpi, 

1997: 40).  “Outsourcing  is  often  thought  of  as  a  cost-cutting  tactic,  but  it  may  cost  the 

same  as  doing  things  yourself;  the  savings  may  be  in  cutting  cycle  or  delivery  times, 

moving  products  faster  or  primarily  in  not  doing  something  wrong”  (Lear-Olimpi,  1997: 

42).  “Companies  often  make  parts  out  of  a  sense  of  corporate  responsibility  -  namely  a 

desire  to  preserve  jobs”  (Venkatesan,  1992:  100).  This  type  of  thinking  can  quickly  put  a 

company  in  financial  trouble.  DoD  is  not  unlike  private  industry  when  it  comes  to 

making  a  decision  whether  to  make  or  buy  a  needed  resource.  Resources  are  limited 

within  both  industry  and  DoD  and  these  limited  resources  must  be  judiciously  expended. 
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The  DoD  must  ensure  that  decisions  regarding  outsourcing  are  made  based  on  sound 
business  practices. 

Companies  report  that  outsourcing  enables  them  to  focus  on  their  core 

competencies;  improve  service  quality,  responsiveness,  and  agility;  obtain  access  to  new 

technologies;  and  employ  more  efficient  business  practices.  Competitive  forces  have 

encouraged  the  creation  of  entire  new  industries  to  meet  the  demand  for  specialized 

services.  “In  1996,  these  service  industries  will  generate  an  estimated  $100  billion  in 

sales"  (Armed  Forces  Information  Service,  1996: 2).  Surveys  performed  by  a  range  of 

organizations  all  document  the  trend  toward  more  outsourcing. 

Some  examples  include:  (1)  A  1994  study  by  Pitney  Bowes 
Management  Services  found  that  77  percent  of  100  Fortune  500 
firms  surveyed  outsourced  some  aspect  of  their  business  support 
services;  (2)  A  1992  study  of  1200  companies  conducted  by  the 
Outsourcing  Institute  found  that  50  percent  of  firms  with  information 
technology  budgets  over  $5  million  are  either  outsourcing  or  actively 
considering  it;  (3)  A  1994  study  conducted  by  KPMG-Peat  Marwick 
of  309  Fortune  1000  companies  found  that  48  percent  outsourced 
warehousing  functions:  and  (4)  A  1994  study  conducted  by  the 
Olsten  Corp.  of  400  firms  found  that  45  percent  outsourced  payroll 
management  functions.  (Armed  Forces  Information  Service,  1996:  7) 


Studies  suggest  that  many  local  governments,  including  those  of  Chicago,  Indianapolis, 
Los  Angeles,  Philadelphia,  and  San  Francisco,  have  used  outsourcing  as  a  way  to  lower 
costs  and  improve  customer  service. 

The  myriad  of  ways  to  outsource  makes  defining  privatization,  the  government 
version  of  make  or  buy,  difficult.  In  its  purest  form,  the  term  refers  to  the  shifting  of  the 
production  of  a  good  or  the  provision  of  a  service  from  the  government  to  the  private 
sector,  often  by  selling  government  owned  assets.  According  to  Elaine  Kamarck,  who 
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heads  Vice  President  Gore’s  National  Performance  Review,  “Outsourcing  means  purely 
divesting  the  government  function”  (Shoop,  1995:  17).  Most  definitions  of  privatization 
are  more  expansive,  including  any  action  that  involves  exposing  the  operations  of 
government  to  the  commercial  market.  It  is  often  difficult  to  tell  where  government  ends 
and  the  private  sector  begins.  The  following  definitions  are  provided  in  an  article  by 
Tom  Shoop  entitled  Going,  Going,  Gone: 

-Selling  Out-  Sales  of  federal  assets.  Example:  Naval  petroleum  Reserves. 

-Service  Shutdown  -  The  government  discontinues  or  gives  up  responsibility  for  a 
service.  Example:  Office  of  Personnel  Management’s  training  operation. 

-Quasi-Government  -  The  netherworld  of  federal  corporations,  government 
sponsored  enterprises  and  quasi-govemment  organizations.  Often  an  intermediate  step 
on  the  road  to  full  privatization.  Examples:  Amtrak,  Postal  Service. 

-Public-Private  Partnerships  -  Any  effort  in  which  government  and  private 
organizations  share  ownership  of  assets  and  operational  responsibilities.  Includes  voucher 
systems,  franchise  arrangements-even  volunteer  programs.  Examples:  Food  stamps. 
Medicare,  and  job-training  programs. 

-Contracting  Out  -  The  most  common  form  of  privatization:  shifting  the 
provision  of  a  good  or  service  to  the  private  sector  by  putting  it  out  for  competitive  bid. 
Examples:  everything  from  construction  of  aircraft  carriers  and  management  of  nuclear 
weapons  facilities  to  provision  of  landscaping  services  (Shoop,  1995:  17). 
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Regulatory  and  Policy  Guidance 

Various  laws  contained  in  the  National  Defense  Authorization  Act  for  fiscal  years 

1988-89  inhibited  DoD's  ability  to  outsource.  Public  Law  100-180,  otherwise  known  as 

the  Nichols  Amendment,  allowed  installation  commanders  to  determine  whether  to 

pursue  outsourcing.  The  Department  of  Defense  Appropriations  Act  for  fiscal  year  1991 

contained  Public  Law  101-511,  which  prohibited  funding  for  lengthy  A-76  studies.  The 

National  Defense  Authorization  Act  for  1993-1994  prohibited  DoD  from  entering  into 

contracts  resulting  from  cost  studies  done  under  OMB  Circular  A-76,  from  October  23, 

1992  to  April  1, 1994  (Armed  Forces  Information  Service,  1996).  Even  though  these 

prohibitions  have  all  lapsed,  a  number  of  provisions  in  Title  10,  chapter  146  continue  to 

have  an  impact  on  the  outsourcing  decision.  They  include: 

-Cost  comparison  studies  are  required  if  outsourcing  impacts  more  than 
10  positions. 

-Congress  must  be  notified  of  the  results  of  any  A-76  studies  involving 
more  than  45  civilians. 

-Annual  reports  to  Congress  are  required  on  outsourcing. 

-DoD  is  prohibited  from  outsourcing  firefighters  or  security  guards  at 
military  installations. 

-Outsourcing  is  prohibited  for  functions  that  are  considered  inherently 
governmental. 

-Military  services  must  maintain  the  capability  to  accomplish  60%  of 
their  depot-level  maintenance  organically. 


Recent  Problems  and  Concerns  Regarding  Outsourcing 

The  NPR  and  challenges  identified  in  Joint  Vision  2001  have  been  the  main 
drivers  for  recent  acquisition  streamlining  legislation  including:  The  Federal  Acquisition 
Streamling  Act  (FAS A)  of  1994,  The  Federal  Acquisition  Reform  Act  (FARA)  of  1996, 
and  recent  DoD  5000.1  and  5000.2  directives.  Previous  annual  budgeting  practices  did 
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not  properly  motivate  government  organizations  to  cut  their  expenses.  If  they  did  not 

spend  their  entire  annual  budget,  then  the  next  year's  budget  was  reduced  accordingly. 

This  promoted  waste  and  massive  spending  sprees  at  the  end  of  the  fiscal  year.  To 

correct  these  inefficiencies,  the  Deputy  Secretary  of  Defense  signed  a  memorandum  on 

February  26, 1996  stating  that  "Resources  saved  through  these  initiatives  during  the  POM 

process  will  not  be  decremented  from  your  out-year  budgets  and  should  instead  be 

applied  to  your  modernization  priorities"  (Armed  Forces  Information  Service,  1996: 25). 

These  pieces  of  acquisition  legislation  streamline  the  acquisition  process,  but  do  not 

entirely  eliminate  the  road  blocks  to  outsourcing  and  privatization.  Title  10  of  the  United 

States  Code  (10  U.S.C.)  still  includes  many  hurdles  to  achieving  the  necessary  funds  to 

meet  the  three  challenges  identified  in  Joint  Vision  2001 : 

Provisions  of  law  that  impede  achieving  the  benefits  of  minimized 
costs  and  requisite  readiness  include:  (1)  Section  2466  of  10  U.S.C. 
requires  60  percent  of  depot  maintenance  to  be  performed  by  Federal 
Employees;  (2)  Section  2464  of  10  U.S.C.  defines  Core  Logistics  Functions 
in  terms  of  workload  performed  at  specified  facilities.  This  creates  an 
artificial  constraint  that  hinders  the  DoD’s  ability  to  effectively  manage  its 
depot  maintenance  facilities;  (3)  Section  2469  of  10  U.S.C.  requires 
public/private  competitions  before  any  workload  in  excess  of  $3  million 
can  be  transferred  to  the  private  sector;  (4)  Section  2461  of  10  U.S.C. 
requires  four  separate  reports  that  DoD  feels  are  unnecessary.  In  addition, 
the  extensive  “how  to”  requirements  create  disincentives  for  DoD  components 
to  pursue  outsourcing.  It  also  restricts  the  use  of  funds  for  cost  comparison 
studies  that  are  not  completed  within  24  months;  (5)  Section  8020  of  the  DoD 
Appropriations  Act,  1996  requires  the  same  detailed  analysis  of  a  function 
involving  10  employees  as  it  does  a  function  involving  1,000  employees; 
and  (6)  Section  317  of  the  National  Defense  Authorization  Act  for  Fiscal  Year 
1987  exempts  specific  installations  for  political  or  other  reasons.  (Armed  Forces 
Information  Service,  1996: 15) 
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Recent  Developments 

“In  the  17-year  period  between  1979  and  1996,  the  Defense  Department  reviewed 
about  90,000  positions  for  transfer  to  the  private  sector;  through  2005,  the  department 
says  it  will  review  229,000  positions”  (Peters,  1999: 20).  DoD  will  compete  more  than 
twice  as  many  positions  between  1997  and  2005  in  one-third  the  time.  Figure  1  shows 
the  number  of  positions  planned  for  A-76  competitions  between  1997  and  2002. 


Positions  Planned  for  A-76  Competition 


Figure  1:  Positions  Planned  for  A-76  Competition  (Peters,  1999:22) 

History  has  shown  that  no  matter  who  wins  the  competition,  jobs  will  be  reduced. 
DoD  competed  5,757  positions  from  October  1995  to  March  1998.  Table  1  shows  that 
while  sixty  percent  of  the  competitions  were  awarded  to  contractors,  85  percent  of  the 
positions  were  lost.  There  was  a  reduction  in  manpower  even  in  those  instances  when  the 
government  was  the  winner  of  the  competition. 
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Table  1 .  A-76  Competition  Results  From  Oct.  1995  to  Mar.  1998  (Peters,  1999: 24) 


Tough  Competition 

Component 

A-76 

Studies 

Completed 

Contractor 

Awarded 

Total 

Positions 

Competed 

Total  Positions 
Reduced 

Army 

3 

2  (67%) 

94 

78 

Navy 

3 

3  (100%) 

154 

124 

m 

■ 

■iim 

■HI 

MIIS  1 

Defense 

Commissary  Agency 

4 

3  (75%) 

95 

91 

Defense  Finance  and 
Accounting  Service 

2 

0 

519 

257 

Total 

53 

32  (60%) 

5,757 

4,221 

Last  year  Congress  passed  the  Federal  Activities  Inventory  Reform  (FAIR)  Act 
which  put  even  more  pressure  on  federal  agencies  to  outsource.  This  Act  requires 
Defense  and  other  executive  agencies  to  publish  annually  a  list  of  activities  that  are  not 
inherently  governmental  but  that  are  performed  by  a  government  source.  This  Act  paves 
the  way  for  agencies  to  contract  even  more  work.  “Agencies  won’t  publish  their 
commercial  activities  list  until  June  30,  but  word  on  the  street  is  that  DoD’s  inventory 
will  be  the  most  aggressive  by  far”  (Peters,  1999: 22). 


F-117  Developments 

In  July  1995,  General  Henry  Viccellio,  Commander,  Air  Force  Materiel 
Command,  cited  the  F-l  17  as  a  candidate  for  “Contractor  Logistics  Support  (CLS)  for 
Life”  to  reduce  total  operating  costs.  This  proposal  eventually  led  to  the  decision  to 
outsource  F-117  sustainment  tasks  that  were  being  performed  by  the  Sacremento  Air 
Logistics  Center.  The  Total  System  Performance  Responsibility  (TSPR)  contract,  signed 
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in  October,  1998,  transitioned  responsibility  for  the  following  tasks  from  the  government 
to  the  contractor:  item  management,  material  management,  warehousing,  transportation, 
supply  support,  sustaining  engineering,  program  management,  technical  orders, 
production  management,  configuration  management,  data  management,  test  and 
evaluation  management,  and  safety.  The  transition  of  these  responsibilities,  in  addition  to 
the  actual  maintenance  the  contractor  was  already  performing,  gave  total  control  and 
responsibility  for  depot-level  maintenance  to  the  contractor. 

Recently,  General  George  Babbitt,  the  Commander  of  AFMC,  and  the  Program 
Executive  Officer  for  fighter/bombers,  General  Bolton,  requested  the  F-l  17  System 
Program  Office  to  do  a  “business  case.”  The  business  case  is  an  effort  to  go  back  to  the 
beginning  before  the  decision  to  outsource  sustainment  tasks  was  made  in  an  effort  to 
validate  the  assumptions  used  in  the  decision.  The  intent  is  to  establish  a  baseline  so  they 
can  compare  projected  savings  with  actual  savings.  The  Program  Office  is  interested  in 
reviewing  the  methodology  and  results  of  this  thesis  effort  to  assist  it  in  completing  their 
business  case. 

Chapter  Summary 

Recent  GAO  Reports  and  warnings  from  the  corporate  sector  about  outsourcing 
have  created  the  need  for  this  research  study  (GAO  Report,  GAO/NSIAD-98-48, 1997). 
The  GAO  report  entitled  "Savings  Achievable  But  Defense  Science  Board's  Projections 
Are  Overstated"  has  sparked  concern  among  Air  Force  leadership  because  many 
organizations  have  included  these  projected  savings  in  their  out-year  budgets.  In 
addition,  in  a  report  to  the  Senate  Armed  Services  Committee  on  October  6th,  1998, 
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Secretary  of  Defense  William  S.  Cohen  stated:  "Without  additional  (base)  closures  we 
will  not  achieve  the  $20  billion  in  projected  savings  in  the  years  where  some  major 
systems  are  scheduled  to  come  on  line"  (Garamone,  1998: 25).  The  GAO  report  and 
Secretary  Cohen’s  statement  are  driving  the  desire  for  Air  Force  leaders  to  quantify  the 
savings  that  they  are  achieving  from  outsourcing.  Although  a  great  deal  has  been  written 
about  why  you  should  outsource,  how  you  should  outsource,  and  when  you  should 
outsource,  there  has  been  nothing  written  on  how  to  measure  and  /or  determine  whether 
or  not  outsourcing  is  a  better  overall  value  than  maintaining  the  in-house  capability.  The 
purpose  of  this  study  is  to  explore  traditional  techniques  in  business  research  to  determine 
whether:  1)  one  can  develop  a  reliable  snapshot  of  how  effective  outsourcing  is  as  an 
alternative  to  organic  solutions;  and  2)  determine  whether  analytical  lessons  learned  from 
this  application  can  be  generalized  and  applied  to  future  valuation  assessments. 
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III.  Methodology 


Chapter  Overview 

As  stated  in  previous  sections  of  this  thesis,  the  goal  of  this  research  is  to 
determine  whether  or  not  outsourcing  of  depot-level  maintenance  is  a  better  value  than 
organically  performed  depot-level  maintenance.  Our  proposition  is  that  outsourcing  the 
F-l  17  depot-level  maintenance  is  a  better  value  than  organically  performed  maintenance. 
A  finding  inconsistent  with  this  proposition  suggests  that  more  research  is  necessary. 
Archival  data  reflecting  cost  and  key  performance  indicators  from  the  last  two  years  were 
compared  to  identical  cost  and  performance  factors  reflecting  contractor  performance 
since  inception  of  the  Total  System  Performance  Responsibility  (TSPR)  contract.  We 
sought  to  measure  perceived  customer  satisfaction  by  comparing  satisfaction  levels 
before  and  after  the  TSPR  contract  was  awarded.  By  comparing  customer  satisfaction 
data  to  the  results  of  the  archival  analysis,  we  should  be  able  to  determine  if  the  Air  Force 
is,  or  is  not,  obtaining  a  better  value  through  outsourcing. 

Research  Approach 

To  test  our  proposition,  we  used  archival  data  to  compare  the  government’s 
performance  to  the  contractor’s  performance.  Technical  performance  was  measured 
against  the  metrics  tracked  by  Air  Combat  Command  (ACC)  with  the  Monthly  Aircraft 
Logistics  Indicators  Report  (9302  Report),  a  report  used  to  measure  logistics  indicators 
for  all  ACC  aircraft.  These  performance  metrics  include:  mission  capable  (MC)  rate, 
total  non-mission  capable  supply  (TNMCS)  rate,  mission  capable  (MICAP)  delivery, 
replacement  spare  parts  (RSP)  kit  fill  rate,  depot  delivery,  depot  quality,  deficiency  report 
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(DR)  response  rate,  and  weapon  system  trainer  (WST)  availability.  These  same 
performance  metrics,  with  the  exception  of  the  MC  rate,  are  contained  in  the  TSPR 
contract  and  are  used  to  gauge  the  contractor’s  performance.  Our  evaluation  of  these 
factors  considered  the  number  of  monthly  sorties  flown.  The  amount  of  award  fee  the 
contractor  receives  is  dependent  upon  how  well  it  performs  against  these  metrics.  We  are 
using  actual  cost  and  performance  data  from  fiscal  years  1997  and  1998.  We  believe 
that,  due  to  the  time  of  contractor  take  over,  age  of  the  fleet,  number  of  sorties  flown,  and 
total  aircraft  in  the  inventory  during  this  period,  that  the  most  common  conditions  exist  to 
make  a  fair  comparison.  Earlier  data  is  available;  however,  we  cannot  verify  its 
reliability  and  it  would  require  the  leveling  of  several  confounds.  It  must  be  noted  that 
the  contractor  has  always  performed  the  actual  hands-on  maintenance  due  to  the  unique 
nature  of  the  aircraft  material.  Prior  to  the  decision  to  have  the  contractor  assume  total 
responsibility  for  depot-level  maintenance,  the  government  controlled  development  and 
acquisition,  program  management,  sustainment,  budgeting,  contracting,  and  security. 
Sustainment  tasks  include  item  management,  material  management,  warehousing, 
transportation,  supply  support,  sustaining  engineering,  program  management,  tech  orders, 
production  management,  configuration  management,  data  management,  test  and 
evaluation  management,  and  safety.  Under  the  TSPR  contract  awarded  1  October  1998 
(beginning  FY  1999),  the  contractor  assumed  all  sustainment  responsibilities. 

To  supplement  the  archival  analysis  of  before  and  after  TSPR  cost  and 
performance  data,  we  supported  our  findings  by  soliciting  customer  satisfaction 
information.  To  examine  the  customer's  perceived  increase  or  decrease  in  value,  we 
questioned  the  customers  using  a  Likert  five-point  comparative  ranking  scale  (Cooper 


and  Emory,  1995).  This  questionnaire  serves  two  purposes.  First,  the  questionnaire 
results  are  key  indicators  of  customer  satisfaction  both  before  and  after  the  TSPR  contract 
was  awarded.  This  comparison  assisted  us  in  our  attempt  to  examine  whether  or  not 
outsourcing  is  a  better  value.  Second,  the  questionnaire  is  an  indicator  of  the  validity  of 
the  performance  indicators  used  to  measure  the  contractor’s  performance.  For  example, 
if  the  archival  data  indicate  that  the  contractor  is  meeting  or  exceeding  all  performance 
indicators,  but  the  customer  is  not  satisfied  with  performance,  this  may  indicate  that  the 
wrong  metrics  are  being  used  to  measure  performance. 

Instrument  Implementation 

The  questionnaire  is  a  short,  thirteen  question  Likert-type  questionnaire  that  was 
directed  at  those  with  knowledge  of  the  depot  supply  process  and  how  it  was  affected  by 
the  award  of  the  TSPR  contract.  These  groups  include  supply,  expeditor,  inspection  and 
acceptance,  and  senior  maintenance  personnel  who  track  or  are  aware  of  the  differences 
in  the  level  of  performance.  The  individuals  are  from  the  same  units  that  provide  input  to 
the  award  fee  board.  Originally,  we  planned  on  selecting  a  random  sample  of  these 
groups,  but  were  unable  to  obtain  a  list  of  all  candidates  available  in  these  groups  due  to 
ongoing  deployment  operations.  The  Commander  of  the  F-l  17  fighter  wing  at  Holloman 
AFB  forwarded  the  questionnaire  by  email  and  tasked  his  leadership  to  forward  it  to  key 
personnel.  The  number  of  personnel  available  to  answer  the  questionnaire  was  limited 
due  to  an  increased  operations  tempo  caused  by  the  deployment  of  unit  personnel  to 
Kosovo.  While  not  random,  we  believe  the  selection  of  participants  is  valid  for  purposes 
of  our  study.  The  distribution  of  questionnaires  to  key  personnel  should  improve  the 
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quality  of  responses.  Once  responses  were  received,  the  data  were  analyzed  to  show  how 
each  value  determinant  affected  the  response  variable.  Because  outsourcing  is  viewed  by 
some  as  a  sensitive  subject,  we  addressed  concerns  of  anonymity  and  fear  of  retribution 
by  ensuring  that  respondents'  names  were  kept  anonymous. 

Data  Collection  Process 

The  questionnaire  is  provided  in  Appendix  B,  and  relates  primarily  to  established 
performance  parameters.  It  also  deals  with  the  perception  of  overall  improved  efficiency, 
quality  and  working  relationship  between  the  depot  and  the  flight-line  maintenance 
personnel.  The  target  population  of  the  questionnaire  was  those  with  knowledge  of  the 
depot  supply  process  and  how  the  award  of  the  TSPR  contract  affected  it.  These 
personnel  had  knowledge  of  factors  relating  to  MC  rates,  TNMCS  rates,  MICAP 
delivery,  RSP  Kit  fill  rates,  depot  delivery,  depot  quality,  DR  response  rates,  and  WST 
availability.  They  are  also  in  the  position  to  determine  if  these  are  the  proper  factors  to 
use  to  measure  the  contractor's  performance. 

Limitations 

The  scope  of  this  study  is  limited  by  several  factors.  First,  the  contractor  has  only 
been  responsible  for  total  system  performance  for  seven  months  and  for  the  last  three 
months  has  been  supporting  deployments  to  Kosovo.  This  is  important  because  under 
TSPR  some  of  the  performance  metrics  are  relaxed  until  the  aircraft  return  from  the 
theatre.  Also,  upon  inception  of  the  contract,  the  contractor  assumed  control  of  a 
warehouse  full  of  government  acquired  spare  parts.  In  addition,  the  contractor  was  not 
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held  accountable  for  performance  metrics  on  aircraft  in  process  during  the  change  over. 
As  a  result,  the  first  quarter  data  does  not  accurately  reflect  the  contractor’s  ability  to 
perform.  Second,  because  of  the  recent  deployments,  operations  tempo  data  normally 
available  is  considered  sensitive  and  therefore  not  accounted  for  in  this  analysis.  Third, 
we  realize  the  historical  cost  data  being  compared  may  not  accurately  reflect  a 
comparison  against  the  government's  most  efficient  organization  (MEO)  costs.  Although 
the  F-l  17  Program  Office  did  accomplish  an  MEO,  it  was  not  used  as  a  cost  baseline 
because  the  decision  was  already  made  to  have  the  jobs  remain  in  California.  Also,  other 
costs,  such  as  employment  relocation  programs,  early  retirement  incentives,  and 
additional  DCMC  involvement,  are  not  accounted  for  in  the  SPO's  estimate  of  the  costs  to 
convert  sustainment  to  contractor  control.  Finally,  we  were  unable  to  analyze  the  impact 
of  spares  funding  levels  during  the  before  and  after  periods,  which  could  have  a 
significant  impact  on  the  ability  of  either  party  to  meet  or  exceed  the  performance 
metrics. 

Chapter  Summary 

This  chapter  provides  an  overview  of  our  research  methodology.  It  explains  our 
research  approach,  instrument  implementation,  data  collection,  and  limiting  factors.  The 
purpose  of  this  chapter  was  to  prepare  the  reader  for  the  detailed  analysis  that  appears  in 
Chapter  4,  and  make  the  reader  aware  of  the  factors  that  limit  this  research  effort. 
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IV.  Analysis  and  Results 


Chapter  Overview 

The  purpose  of  this  analysis  is  to  examine  if  outsourcing  of  the  F-l  1 7  sustainment 
effort  is  a  better  value  to  the  government  than  organic  support.  The  value  of  this 
information  is  that  the  AF  decision-maker  will  have  a  single  case  study  that  will  provide 
information  useful  for  making  future  outsourcing  decisions. 

The  data  are  presented  in  the  following  sequence.  First,  a  detailed  cost  analysis  is 
presented  examining  the  actual  and  perceived  financial  value  to  the  AF.  Next,  we  present 
results  comparing  performance  metrics.  This  is  followed  by  a  discussion  of  the  customer 
satisfaction  questionnaires  and  expert  interviews.  Finally,  we  present  a  triangulation  of 
the  cost,  performance,  and  customer  satisfaction  information. 

Cost  Comparison 

A  detailed  analysis  comparing  costs,  MC  rates,  TNMCS  rates,  MICAP  delivery, 
RSP  Kit  fill  rate,  depot  delivery,  depot  quality,  DR  response,  and  WST  availability  before 
and  after  sustainment  responsibilities  were  transferred  to  the  contractor  was  performed. 
Actual  cost  and  performance  data  from  fiscal  years  1997  and  1998  were  compared  to 
contractor  performance  data  from  October  1998  through  April  1999.  The  results  of  this 
analysis  were  compared  to  the  results  of  the  customer  satisfaction  questionnaire  and 
expert  interviews. 

Figure  2  and  Table  2  are  graphical  and  tabular  representations  of  the  program 
office’s  estimate  of  savings  resulting  from  the  transfer  of  sustainment  responsibilities 
from  the  government  to  a  contractor.  The  figure  identifies  two  potential  savings  profiles; 
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one  assumes  stable  F-l  17  program  funding  and  the  other  assumes  unstable  funding.  As 
of  the  completion  date  of  this  report,  stable  funding  beyond  FY99  has  not  been 
authorized.  The  “requirement  without  TSPR”  costs  are  an  estimate  of  what  it  would  have 
cost  the  government  to  accomplish  sustainment  efforts  without  the  TSPR  contract.  The 
amounts  are  calculated  using  FY98  actual  government  sustainment  costs  as  a  baseline, 
escalated  three  percent  per  year  to  cover  projected  inflation.  The  F-l  17  Program  Office 
made  a  business  decision  to  not  escalate  Government  personnel  costs.  This  decision 
tends  to  understate  projected  savings. 


TSPR  vs  Without  TSPR 


£ 
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Fiscal  Year 
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Figure  2.  Program  Office’s  Estimate  of  Savings  From  TSPR  (F-l  17  Program  Office) 
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Table  2.  Program  Office's  Estimate  of  Savings  From  TSPR  (F-117  Program  Office) 


TSPR  vs  Without  TSPR 

Cost  Comparison  vs  Inflation 

Fiscal  Year 

Rqmnts 
without  TSPR 

SPO  West  Manning 

Tot  Rqmnts 
without  TSPR 

TSPR  without 
Stable  $ 

TSPR  with 
Stable  $ 

FY96 

191.8 

11.58 

203.4 

FY97 

11.58 

196.5 

FY98 

187.3 

11.64 

198.9 

192.9 

11.64 

204.5 

185.3 

181 

198.7 

11.64 

210.3 

197.4 

194.7 

204.7 

11.64 

mmm 

193.7 

210.8 

11.64 

222.4 

209.2 

199.7 

11.64 

228.7 

215.2 

201.2 

223.6 

11.64 

235.2 

•  225.7 

208.6 

FY05 

230.4 

11.64 

242 

233.5 

216.8 

FY06 

237.3 

11.64 

248.9 

240.3 

232.4 

Tot  99-06 

1715.5 

93.1 

1808.6 

1628.1 

Savings 

TSPR  w/o  Stable  $ 

99-06 

TSPR  w/ Stable  $ 

180.5 

180.5 

This  reduction  is  an  estimate  of  the  reduction  of  ALC 
personnal 

-2.94 

-177.6 

From  Table  2,  above,  the  AF  projected  savings  is  $177.6  million  over  the  life  of 
the  sustainment  contract.  This  level  of  projected  savings  assumes  the  government  is  able 
to  fully  fund  the  TSPR  contract  so  the  contractor  can  negotiate  long-term  contractual 
relationships  with  their  subcontractors  and  can  implement  other  long-term  cost  savings 
initiatives.  Without  full  funding  the  contract  would  need  to  be  renegotiated  resulting  in  a 
loss  of  savings.  The  AF  calculated  the  projected  savings  by  developing  an  estimate  of 
what  it  would  have  cost  the  government  to  provide  the  identical  level  of  support  and 
compared  this  estimate  to  the  actual  contract  amount  at  target  price.  The  AF  estimate 
was  developed  by  escalating  the  AF  sustainment  costs  for  fiscal  year  1998  by  three 


28 


percent  per  year  projected  over  the  life  of  the  contract.  The  savings  are  calculated  by 
subtracting  the  actual  contract  amount,  at  target  price,  from  the  government’s  estimate  of 
what  it  would  have  cost  the  AF  to  accomplish  this  same  level  of  work.  The  only  direct 
comparison  that  can  be  made  is  between  the  estimated  cost  for  the  government  to  provide 
sustainment  services  during  FY99  if  the  TSPR  contract  was  not  awarded  and  the 
projected  contract  cost  for  FY99.  The  government  estimate  was  based  on  the  assumption 
that  the  same  level  of  service  would  be  provided  by  the  government.  It  must  be  noted 
that  only  seven  months  of  cost  history  is  available  from  the  TSPR  contract.  The  amount 
used  for  comparison  purposes  is  the  estimate  to  complete  provided  by  the  TSPR 
contractor.  The  projected  cost  under  TSPR  based  on  the  estimate  to  complete  for  fiscal 
year  1999  equates  to  $177.6  million.  The  contract  contains  a  provision  that  encourages 
contractor  efficiencies  by  sharing  any  savings  below  the  $181  million  target  price  50/50 
between  the  government  and  the  contractor.  The  contractor  is  projecting  a  $3.4  million 
under-run,  of  which  the  government  share  is  $1 .7  million.  The  savings  to  the  government 
for  FY99  equates  to  $25.2  million.  This  represents  the  difference  between  the 
government  estimate  of  what  it  would  have  cost  to  perform  the  effort  minus  the  contract 
price  at  target,  plus  the  government’s  share  of  the  under-run. 

In  conclusion,  the  TSPR  contract  appears  to  offer  significant  financial  savings  to 
the  government  as  compared  to  what  it  would  have  cost  the  government  to  accomplish 
sustainment  tasks  at  the  same  level  of  support.  Although  there  is  no  guarantee  that 
contractor  cost  performance  will  be  better  or  worse  during  the  life  of  the  contract;  the 
analysis  provides  an  initial  look  at  contractor  cost  performance  from  which  a  projection 
of  future  savings,  if  any,  can  be  estimated. 
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Technical  Performance 


Technical  performance  was  gauged  by  comparing  the  performance  before 
sustainment  responsibility  was  transferred  to  the  contractor  with  performance  after 
transfer.  Performance  was  measured  against  pre-established  standards.  The  same 
standards  were  applied  with  two  exceptions.  The  TSPR  contractor  is  not  held 
accountable  for  meeting  the  MC  rate  because  many  of  the  factors  influencing  this  rate  are 
under  government  control.  Also,  the  standard  for  Total  Non-Mission  Capable  Supply 
(TNMCS)  was  reduced  from  seven  percent  to  five  percent  for  contractor  performance. 
The  contractor  is  held  to  a  higher  standard  because  the  AF  retained  responsibility  for 
parts  the  government  must  supply. 

Table  3  presents  a  comparison  of  the  government’s  performance  for  fiscal  years 
1997  and  1998  and  contractor  performance  for  the  first  seven  months  of  fiscal  year  1999 
under  TSPR.  The  comparison  is  based  on  the  performance  metrics  in  the  contract  with 
the  exception  of  the  MC  rate. 
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Table  3:  Performance  Comparison  Non-TSPR  vs  TSPR  (F-l  17  Performance  Metrics) 


Table  4  compares  the  pre-TSPR  average  performance  levels  for  fiscal  years  1997 
and  1998  with  the  contractor’s  average  performance  for  the  first  seven  months  of  fiscal 
year  1 999.  The  analysis  of  the  data  in  Tables  3  and  4  indicates  that  from  a  technical 
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perspective,  performance  was  generally  satisfactory  both  prior  to  and  after  transfer  of 
sustainment  responsibility  to  the  contractor.  The  performance  indicators  for  the  seven 
months  since  the  TSPR  contract  was  signed  are  higher  for  four  of  the  seven  performance 
indicators  used  to  measure  the  contractor  under  the  TSPR  contract,  and  tied  for  a  fifth. 
Note  that  depot  quality  is  a  single  performance  indicator;  however,  major  and  minor 
discrepancies  have  been  separated  for  ease  of  comparison. 


Table  4:  Average  Performance  Levels 


Metric 

Avg97 

Avg98 

Avg  99 

MC  Rate  (%) 

■ 

1111 

■ 

78.92 

83.56 

TNMCS  Rate  (%) 

4.39 

5.98 

HlHEilHHH 

MICAP  Delivery(Hrs) 

61.74 

63.31 

■HUliHi 

RSP  Fill  Rate  (%) 

98.74 

wmm 

99.03 

Depot  Delivety  (Days) 

N/A 

1.79 

sn 

Depot  Quality  Majors 

N/A 

IHlliH 

2.19 

Depot  Quality  Minors 

N/A 

■mu 

2.49 

DR  Response  (Days) 

0.11 

0:00 

HUbSHI 

WST  Availability  (%) 

■  JiM 

99.80 

The  different  lengths  of  the  performance  periods  pose  concerns,  but  by  comparing 
sortie  rates  we  can  reasonably  correlate  the  two  data  points.  Figure  3  shows  the  level  of 
sorties  flown  during  the  periods  we  are  comparing.  With  the  exception  of  September 
1 997,  the  sortie  rates  have  been  relatively  stable  each  month.  The  data  for  that  month  is 
much  lower  because  the  fleet  was  grounded  after  an  aircraft  crashed.  Figure  3  is 
significant  because  it  indicates  that  sustainment  tasks  required  during  fiscal  years  97  and 
98  and  the  first  seven  months  of  fiscal  year  99  are  based  on  comparable  sortie  rates. 
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F-117  Sorties  Flown 
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Figure  3:  F-117  Sorties  Flown  by  Fiscal  Year 


From  Figure  3  above,  the  average  number  of  monthly  sorties  for  FY97  is  613 
(excluding  September),  555  for  FY98,  and  585  for  FY99.  The  results  and  analysis  of 
each  performance  indicator  follow. 

Mission  Capable  Rate :  The  standard  is  80%.  Several  factors  comprise  this  rate, 
some  of  which  are  not  under  contractor  control.  Nonetheless,  we  are  including  this 
performance  indicator  in  our  analysis  because  many  of  the  metrics  for  which  the 
contractor  is  held  accountable  significantly  impact  this  rate.  The  average  mission  capable 
rate  for  the  first  seven  months  under  TSPR  is  83.56%.  This  rate,  statistically,  is  not 
significantly  different  from  the  pre-TSPR  FY97  rate,  but  is  a  5.9%  improvement  over  the 
observed  performance  during  FY98.  The  monthly  rate  under  TSPR  exceeds  the  standard 
on  average;  however,  the  standard  was  not  met  for  the  month  of  April  1 999.  We  cannot 
determine  if  the  failure  to  meet  standard  was  affected  by  contractor  performance.  A 


33 


number  of  F-l  17’s  are  deployed  to  Kosovo,  significantly  stretching  the  supply  line,  and 
more  than  likely  having  an  impact  on  this  factor. 

Total  Non  Mission  Capable  Supply:  The  standard  set  by  ACC  is  7%.  The 
standard  under  TSPR  is  5%  because  the  government  retains  responsibility  for  furnishing 
parts  managed  by  the  other  services  or  the  Defense  Logistics  Agency.  The  average 
monthly  rate  under  TSPR  is  2.69%,  which  easily  exceeds  the  standard;  however,  the 
contractor  failed  to  satisfy  the  standard  during  the  month  of  April  1 999.  Again,  this 
failure  could  be  attributed,  at  least  in  part,  to  the  Kosovo  deployment.  The  rate  under 
TSPR  exceeds  the  5%  standard  by  46.2%,  the  pre-TSPR  rate  for  FY97  exceeds  the  7% 
standard  by  37.3%,  and  the  FY98  rate  exceeds  the  7%  standard  by  14.6%. 

MICAP  Delivery:  The  standard  is  72  hours.  The  average  monthly  performance 
under  TSPR  of  38.57  hours  is  a  37.5%  improvement  over  the  pre-TSPR  FY97  rate  and  a 
39%  improvement  over  FY98  performance.  The  contractor  has  far  exceeded  the  standard 
of  72  hours  every  month  (except  for  the  first  month);  however,  it  must  be  noted  that  the 
contractor  was  not  held  accountable  for  aircraft  work  in  process  until  February  1999.  A 
significant  increase  in  the  number  of  hours  to  deliver  during  the  month  of  April  1999  was 
thought  to  be  because  of  one  particular  part.  Delivery  time  is  expected  to  drop  again  in 
May. 

RSP  Kit  Fill  Rate:  The  standard  for  this  performance  indicator  is  96%.  The 
average  monthly  RSP  Kit  fill  rate  under  TSPR,  99.03%,  exceeds  the  96%  standard. 

There  is  no  statistically  significant  difference  between  performance  under  TSPR  and  pre- 
TSPR.  The  contractor  easily  met  the  standard  each  of  the  seven  months. 
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Depot  Delivery:  The  standard  for  this  performance  indicator  is  1  day.  The 
performance  under  TSPR  exceeded  the  standard  each  of  the  seven  months.  Performance 
under  TSPR  exceeded  the  pre-TSPR  FY98  performance  by  an  average  of  1.79  days  per 
month.  The  government's  performance  exceeded  the  standard  all  but  the  first  two 
months.  It  took  14  days  to  deliver  in  October  and  7.5  days  to  deliver  in  November;  all 
other  months  were  zero.  Data  is  not  available  for  this  performance  indicator  for  FY97. 

Depot  Quality:  The  standard  for  depot  quality  allows  for  four  major,  and  ten 
minor  discrepancies.  Although  there  is  no  statistically  significant  difference  between 
performance  under  TSPR  and  performance  during  FY98  in  regard  to  major  discrepancies, 
performance  under  TSPR  exceeded  the  standard  each  of  the  seven  months.  Performance 
under  TSPR  also  exceeds  the  standard  of  ten  minor  discrepancies  each  of  the  seven 
months;  however,  performance  under  TSPR  for  minor  discrepancies  was  74.1%  below 
the  FY98  performance.  Data  is  not  available  for  FY97  for  this  performance  indicator. 

Delinquent  Deficiency  Reports:  The  standard  for  this  performance  indicator  is  5 
days.  The  performance  under  TSPR  exceeded  the  standard  each  of  the  seven  months. 
There  is  no  statistically  significant  difference  between  performance  under  TSPR  and 
performance  during  FY97  and  FY98. 

WST Availability:  The  standard  for  Weapon  System  Trainer  availability  is  97%. 
The  performance  under  TSPR  exceeded  this  standard  each  of  the  seven  months.  The 
performance  level  under  TSPR  was  100%  every  month  except  April  1999,  when 
availability  dropped  slightly  to  99.9%.  There  is  no  statistically  significant  difference 
between  performance  under  TSPR  and  performance  during  FY97  and  FY98. 
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Looking  only  at  the  seven  performance  indicators  used  to  measure  the  TSPR 
contractor’s  performance,  it  would  appear  that  there  is  little  difference  between  average 
monthly  performance  before  and  after  TSPR.  We  did  not  attempt  to  gauge  the 
importance  of  the  performance  indicators;  however,  it  should  be  noted  that  performance 
under  TSPR  was  lower  for  depot  quality  than  during  the  FY98  pre-TSPR  period.  In 
addition  the  mission  capable  rate  under  TSPR  is  below  that  experienced  during  FY97,  but 
higher  than  that  experienced  during  FY98.  In  conclusion,  it  appears  that  technical 
performance  is  not  the  controlling  factor  to  determine  if  outsourcing  sustainment  tasks  for 
the  F-l  17  is  a  better  value  than  organic  support.  Without  the  added  value  of  significant 
cost  savings,  the  decision  to  outsource  could  not  be  supported. 

Questionnaire  Results 

The  customer  satisfaction  questionnaire  serves  two  purposes.  First,  the 
questionnaire  results  are  key  indicators  of  customer  satisfaction  both  before  and  after  the 
TSPR  contract  was  awarded.  This  assisted  us  in  our  attempt  to  examine  whether  or  not 
outsourcing  is  a  better  value.  Second,  the  questionnaire  is  an  indicator  of  the  validity  of 
the  performance  indicators  used  to  measure  the  contractor’s  performance.  For  example, 
if  the  archival  data  indicate  that  the  contractor  is  meeting  or  exceeding  all  performance 
indicators,  but  the  customer  is  not  satisfied  with  performance,  this  may  indicate  that  the 
wrong  metrics  are  being  used  to  measure  performance. 

Our  original  plan  was  to  telephonically  interview  a  randomly  selected  sample 
from  the  total  population  of  F-l  17  maintenance  personnel.  The  Logistics  Commander  of 
the  49th  Logistics  Group  informed  us  that  the  hands-on  maintenance  technicians  would 
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have  little,  if  any,  insight  into  the  differences  in  the  performance  indicators  under  TSPR 
compared  to  pre-TSPR.  In  addition,  the  Kosovo  deployment  and  resultant  increase  in 
operations  tempo  has  greatly  increased  the  workload  of  F-l  17  maintenance  personnel, 
further  limiting  our  access  to  them.  He  proposed  that  we  provide  the  questionnaire  to 
senior  level  individuals  only.  We  suggested  he  include  personnel  from  supply,  quality 
assurance,  and  expeditors  along  with  the  senior  maintenance  personnel.  As  a  result,  the 
Commander  provided  the  questionnaire  to  all  of  the  senior  personnel  from  those  four 
groups.  The  responses  were  gathered  and  forwarded  to  us  via  mail. 

The  questionnaire  consists  of  thirteen  questions.  The  first  five  questions  capture 
demographic  information  such  as  rank,  job  title,  years  of  experience  working  on  the 
F-l  17,  education  level,  and  career  skill  level.  The  next  seven  questions  measure  the 
difference,  as  perceived  by  the  customers,  in  the  performance  indicators  used  to  measure 
both  organic  support  prior  to  TSPR  and  contractor  support  under  the  TSPR  contract.  The 
final  question  is  intended  to  examine  if  the  performance  indicators  presently  used  to 
assess  performance  under  TSPR  are  the  best  indicators  of  performance  and  to  elicit 
suggestions  for  improving  the  performance  indicators. 

A  total  of  30  questionnaires  were  returned.  Not  all  questions  relating  to  the 
performance  indicators  were  answered  on  each  questionnaire.  The  number  of  responses 
range  from  20  responses  (out  of  30)  on  question  9  to  a  maximum  of  25  responses  on 
questions  6  and  7.  We  can  only  speculate  that  respondents  failed  to  answer  certain 
questions  because  they  lacked  insight  into  performance  in  relation  to  the  performance 
indicators.  This  is  evidenced  by  some  of  the  responses  to  question  13. 
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The  analysis  begins  with  question  6,  which  is  the  first  question  that  measures  the 
customer’s  perception  of  the  difference  in  the  quality  of  service  received  under  TSPR 
compared  to  pre-TSPR.  The  analysis  examines  the  results  of  each  questionnaire  response 
with  respect  to  the  performance  indicators  to  determine  whether  or  not  the  customer’s 
perception  matches  the  archival  data  collected.  We  examine  the  responses  of  the  entire 
group  first.  If  there  are  discrepancies  between  the  questionnaire  responses  and  the 
archival  data,  we  will  analyze  other  factors  such  as  education  level,  skill  level,  and  the 
number  of  years  of  experience  working  on  the  F-l  17  to  see  if  there  is  any  dependant 
variable  relationship. 

MC  Rate-.  The  questionnaire  did  not  ask  the  perceived  difference  in  MC  rates 
because  performance  in  the  other  areas  being  measured  determines  the  MC  rate.  In 
addition,  the  contractor  is  not  held  accountable  for  this  rate  because  many  of  the  factors 
influencing  this  rate  are  under  government  control.  By  analyzing  the  results  of  each 
question  and  comparing  it  to  the  archival  data  we  can  reach  conclusions  regarding 
whether  or  not  contractor  performance  impacts  the  MC  rate. 

TNMCS  Rate:  The  archival  analysis  indicates  that  performance  under  TSPR 
exceeds  the  standard  and  is  higher  than  performance  during  FY97  and  FY98.  Of  the  30 
respondents  to  the  questionnaire,  1  reported  that  performance  was  much  better,  1 1 
reported  somewhat  better,  1 1  reported  no  change,  2  reported  somewhat  worse,  and  5 
failed  to  respond.  The  results  of  the  questionnaire  indicate  that  the  customers  perceive 
the  contractors  performance  to  be  equal  to  or  better  than  performance  prior  to  TSPR  for 
this  performance  indicator. 
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MICAP  Delivery:  The  archival  analysis  indicates  that  performance  under  TSPR 
reduced  the  delivery  time  by  more  than  half  for  the  months  of  November  through  March, 
FY99.  The  responses  to  the  questionnaire  for  this  metric  show  that  of  the  30  respondents 
6  reported  a  marginal  decrease  in  performance,  14  reported  no  change,  5  reported 
somewhat  of  an  increase  in  performance,  and  5  failed  to  respond.  The  results  of  the 
questionnaire  indicate  that  19  of  the  25  customers  that  responded  to  this  question 
perceive  the  contractors  performance  to  be  equal  to  or  better  than  performance  prior  to 
TSPR  for  this  performance  indicator.  However,  it  must  be  noted  that  the  contractor 
taking  possession  of  the  government’s  spare  parts  warehouse  may  have  affected  the  first 
quarter  data. 

RSP  Kit  Fill  Rate:  The  archival  analysis  indicates  that  performance  under  TSPR 
exceeds  that  experienced  prior  to  TSPR  during  FY  97,  but  falls  slightly  short  of  the  F  Y 
98  performance  rate.  Performance  under  TSPR  has  exceeded  the  standard  each  of  the 
seven  months.  The  responses  to  the  questionnaire  for  this  metric  show  that  of  the  30 
respondents,  1  indicates  much  better  performance,  6  indicate  somewhat  better 
performance,  13  reported  no  change,  4  reported  somewhat  worse  performance,  and  6 
failed  to  respond.  The  results  of  the  questionnaire  indicate  that  20  of  the  24  customers 
that  responded  to  this  question  perceive  that  performance  under  TSPR  to  be  equal  to  or 
better  than  performance  prior  to  TSPR  for  this  performance  indicator. 

Depot  Delivery:  The  archival  analysis  indicates  that  performance  under  TSPR 
exceeds  the  pre-TSPR  FY  98  performance  and  exceeds  the  standard  for  each  of  the  seven 
months.  Data  are  not  available  for  this  performance  indicator  for  FY97.  The  responses 
to  the  questionnaire  for  this  metric  show  that  of  the  30  respondents,  20  reported  no 
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change,  1  reported  somewhat  of  an  increase  in  delivery  time,  and  9  failed  to  respond. 

The  results  of  the  questionnaire  indicate  that  20  of  the  21  customers  that  responded  to  this 
question  perceive  no  change  in  performance  for  this  performance  indicator. 

Depot  Quality:  The  archival  analysis  indicates  that  performance  under  TSPR 
exceeds  the  standard;  however,  it  is  slightly  below  the  pre-TSPR  performance  during  FY 
98.  FY  97  data  was  not  available  for  this  metric.  Of  the  30  respondents  to  the 
questionnaire,  5  reported  a  greatly  decreased  number  of  discrepancies,  1  reported 
somewhat  of  a  decrease,  9  reported  no  change,  5  reported  somewhat  of  an  increase  in  the 
number  of  discrepancies,  and  10  failed  to  respond.  The  results  of  the  questionnaire 
indicate  that  15  of  the  20  perceived  equal  or  better  performance  under  TSPR;  however,  5 
of  the  20  respondents  reported  a  degradation  of  performance. 

Delinquent  Deficiency  Reports:  The  archival  analysis  indicates  that  performance 
under  TSPR  exceeds  the  standard  each  of  the  seven  months.  This  performance  was  equal 
to  the  FY  98  pre-TSPR  performance  and  significantly  better  than  FY  97  performance.  Of 
the  30  respondents  to  the  questionnaire,  1  reported  a  greatly  decreased  number  of 
delinquent  deficiency  reports,  3  reported  somewhat  of  a  decrease,  14  reported  no  change, 

3  reported  somewhat  of  an  increase  in  the  number  of  delinquent  reports,  and  9  failed  to 
respond.  The  results  of  the  questionnaire  indicate  that  18  of  the  21  perceived  equal  or 
better  performance  under  TSPR. 

WST Availability:  The  archival  analysis  indicates  that  the  WST  availability  under 
TSPR  exceeds  the  standard  each  of  the  seven  months.  Availability  under  TSPR  slightly 
exceeded  the  pre  TSPR  availability  of  FY97  and  FY98;  however,  the  FY97  and  FY98 
availability  also  exceeded  the  standard.  We  did  not  survey  the  group  on  this  performance 
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metric  because  it  was  believed  that  they  would  not  have  insight  into  the  availability  of  the 
trainer. 

Overall  Maintenance  Relationship:  This  is  not  one  of  the  measured  performance 
indicators  nor  is  archival  data  available  to  support  this  analysis;  however,  we  believe  the 
working  relationship  between  the  49th  Logistics  Squadron  and  the  place  of  depot  repair  is 
a  significant  indicator  of  perceived  customer  satisfaction.  Of  the  30  respondents  to  the 
questionnaire,  1  reported  a  greatly  improved  relationship,  10  reported  the  relationship 
was  somewhat  improved,  10  reported  no  change,  1  reported  the  relationship  was 
somewhat  worse,  and  8  failed  to  respond.  The  results  of  the  questionnaire  indicate  that 
21  of  the  22  respondents  perceived  an  equal  or  better  working  relationship  between  the 
two  organizations  under  TSPR. 

The  final  question  sought  to  determine  if  the  correct  performance  indicators  are 
being  measured.  The  results  of  the  questionnaire  produced  no  significant  responses. 

Only  5  respondents  answered  the  question  and  none  of  them  indicated  any  significant 
changes  to  the  existing  metrics.  One  did  indicate  that  deployment  initiatives  should  be 
considered  while  another  said  the  metrics  were  good,  but  that  the  survey  had  come  about 
6  months  too  early.  The  respondent  stated  that  “some  indicators  have  not  had  enough 
time  to  be  properly  surveyed.” 

Table  5  summarizes  the  respondent's  answers  to  the  questionnaire.  The  following 
are  the  possible  answers  for  each  value  question. 

Questions:  6  &  8  7. 9. 10, 11  12 

1  Much  Better  Greatly  Decreased  Greatly  Improved 

2  Somewhat  Better  Somewhat  Decreased  Somewhat  Improved 

3  No  Change  Has  Not  Changed  Has  Not  Changed 
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4  Somewhat  Worse  Somewhat  Increased  Somewhat  Worse 

5  Much  Worse  Greatly  Increased  Much  Worse 

The  following  5  responses  were  received  on  question  13: 

-  Respondent  1 3  stated  “need  to  look  at  deployment  initiatives  -  there  are  none.” 

-  Respondent  19  stated  “performance  indicators  are  good,  perhaps  the  survey  is  about 
6-months  early!  Some  indicators  have  not  had  enough  time  to  be  properly  surveyed.” 

-  Respondent  24  suggested  we  look  at  “repair  times  at  the  depot  (how  long  it  takes 
them  to  repair  an  asset).  We  should  also  watch  depot  repair  costs,”  and  stated 
“Overall,  I  am  satisfied  with  the  support  I  have  received.” 

-  Respondent  25  suggested  we  look  at  “Depot  repair  times,  repair  costs,  and  total 
turnaround  time.”  They  also  stated  “all  in  all  I  think  we  can  make  money  with  the 
TSPR  system.” 

-  Respondent  28  stated  “haven’t  been  here  before  Oct.  9. 1  don’t  really  know  what 
TSPR  is  or  what  it  does  for  me.  Good  job  in  meeting  surge  requirements  for  example 
heat  shields.” 
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Table  5:  Customer  Satisfaction  Questionnaire  Responses 


Expert  Interviews 

Each  interviewee  was  asked  to  provide  their  opinion  on  the  three  investigative 
questions.  The  three  investigative  questions  are: 

(1)  Are  the  contract  performance  metrics  good  indicators  of  better  valued 
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(2)  Has  the  decision  to  outsource  F-l  17  depot-level  maintenance  provided  a  better  value 
to  the  Air  Force? 

(3)  If  outsourcing  F-l  17  depot  maintenance  is  an  equal  or  better  value,  do  we  still  want 
to  give  up  the  potential  core  capability? 

The  intent  of  these  interviews  is  to  obtain  information  on  the  intangible  benefits  or 
detriments  to  outsourcing  the  F-l  17  sustainment  effort.  Key  senior  individuals  with 
knowledge  of  F-l  17  depot  support  were  interviewed.  A  summary  of  their  comments 
follows. 

In  general  each  of  the  interviewees  stated  that  the  metrics  used  to  measure 
performance  under  the  TSPR  contract  are  proper  measures  of  a  better  value,  except  they 
did  not  account  for  the  intangibles  like  responsiveness.  In  most  instances,  the 
interviewees  believed  that  responsiveness  is  significantly  improved  under  the  TSPR 
contract. 

All  of  the  interviewees  indicated  that  outsourcing  the  F-l  17  sustainment  effort  has 
resulted  in  a  better  value  to  the  Air  Force. 

On  the  question  of  core  capability,  all  of  the  interviewees  stated  that  the 
government  has  lost  very  little  core  capability  due  to  the  fact  that  the  contractor  has 
always  performed  the  maintenance  on  this  aircraft.  However,  they  stated  that  we  did  lose 
some  core  capability  in  sustaining  engineering  and  asset  management.  The  consensus 
was  that  the  loss  of  the  core  capability  was  small  when  compared  to  the  projected  $177.6 
million  financial  savings  over  the  life  of  the  TSPR  contract. 
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Triangulation 

Data  related  to  contractor  performance  was  collected  from  three  sources:  archival, 
customer  satisfaction  questionnaires  and  expert  interviews.  Our  analysis  of  the  archival 
data  establishes  that  performance  under  the  TSPR  contract  has  been  equal  to  or  better 
than  performance  prior  to  the  award  of  the  TSPR  contract  during  FY97  and  FY98,  when 
measured  by  the  performance  indicators  described  elsewhere  in  this  report.  Our  analysis 
of  the  customer  satisfaction  questionnaire  results  also  indicates  that  the  customer  has 
found  performance  under  TSPR  to  be  equal  to  or  better.  And  finally,  the  data  gained 
through  the  expert  interviews  support  the  analysis  of  the  archival  data  and  the  customer 
satisfaction  questionnaire  results.  In  conclusion,  it  appears  that  technical  performance 
alone  is  not  the  controlling  factor  to  determine  if  outsourcing  sustainment  tasks  for  the  F- 
1 17  is  a  better  value  than  organic  support. 

In  the  case  of  the  F-l  17,  cost  savings  appear  to  be  the  primary  determinant  of 
whether  or  not  outsourcing  is  a  better  overall  value  to  the  Air  Force.  Current  projections 
indicate  savings  of  $25.2  million  for  FY99  alone.  Savings  for  the  life  of  the  TSPR 
contract  are  estimated  to  be  $177.6  million.  This,  coupled  with  the  intangibles  like 
improved  responsiveness  and  improved  working  relationships  identified  during  the  expert 
interviews,  appears  to  show  that  outsourcing  the  F-l  17  is  a  better  value  to  the  Air  Force. 
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V.  Summary  and  Conclusions 


Chapter  Overview 

The  Department  of  Defense  must  reduce  its  cost  of  support  services  if  it  is  to  meet 
national  security  goals  as  long  as  personnel  and  budgets  continue  to  decline.  Army 
Secretary  Louis  Caldera,  Navy  Secretary  Richard  Denzig  and  Acting  Air  force  Secretary 
F.  Whitten  Peters,  in  a  joint  letter,  said  the  BRAC  process  is  “the  only  tool  we  have 
available  to  divest  ourselves  of  unneeded  infrastructure,  consolidate  missions  and  free 
funds  to  improve  priority  programs  on  the  scale  that  we  know  is  required”  (Garamone, 
1999: 13).  In  addition,  the  “24-star  letter”  signed  by  Chairman  Army  Gen.  Henry 
Shelton,  Vice  Chairman  Air  Force  Gen.  Joseph  Ralston,  and  service  chiefs  Army  Gen. 
Dennis  Reimer,  Marine  Corps  Gen.  Charles  Krulak,  Navy  Adm.  Jay  Johnston  and  Air 
Force  Gen.  Michael  Ryan  noted  that  studies  show  that  DoD  has  “23  percent  excess 
capacity”(Garamone,  1999:  13).  The  letter  goes  on  to  say  “BRAC  is  the  single  most 
effective  tool  for  the  services  to  realign  their  infrastructure  to  meet  the  needs  of  changing 
organizations  and  to  respond  to  new  ways  of  doing  business.  No  other  initiative  can 
substitute  for  BRAC  in  terms  of  ability  to  reduce  and  reshape  infrastructure.  Simply 
stated,  our  military  judgement  is  that  further  base  closures  are  absolutely  necessary” 
(Garamone,  1999:  13).  In  spite  of  these  statements  and  recommendations  by  our  military 
leaders,  attempts  at  obtaining  savings  for  modernization  by  closing  bases  and  reducing 
unnecessary  infrastructure  have  met  with  considerable  resistance.  This  avenue  of  savings 
appears  to  be  a  questionable  source.  At  least  for  the  short  term,  outsourcing  and 
privatization  appear  to  be  the  methods  of  choice  for  reducing  support  costs. 
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Previous  chapters  have  identified  DoD’s  plans  for  outsourcing  over  the  next  five 
years.  Organizations  have  reduced  their  future  year’s  budget  requests  in  anticipation  of 
the  projected  savings  resulting  from  these  outsourcing  projections.  While  statistics  show 
that  manning  drops  even  when  the  government  wins  the  competition,  there  is  no  proof 
that  direct  savings  will  result.  Recent  GAO  reports  indicate  that  the  government 
activities  involved  in  outsourcing  have  failed  to  accurately  assess  the  true  costs  of 
outsourcing,  and  have  likely  overestimated  projected  savings.  Our  study  has  found  that 
in  the  case  of  the  F-l  17,  there  was  little  research  to  determine  if  the  projected  savings  are 
real.  The  costs  to  downsize  the  government  workforce  were  not  considered  because 
McClellan  Air  Force  base,  where  the  depot  was  located,  was  slated  for  closure. 

This  chapter  reviews  the  results  of  our  research  with  respect  to  the  investigative 
questions  first  introduced  in  Chapter  1.  First,  it  discusses  whether  the  performance 
indicators  being  used  to  measure  contractor  performance  are  valid.  Next,  it  examines  if 
outsourcing  the  F-l  1 7  sustainment  tasks  have  led  to  a  better  overall  value  to  the  Air 
force.  Finally,  if  the  answer  to  the  second  question  is  yes,  it  examines  if  the  additional 
value  to  the  Air  Force  justifies  the  loss  of  control  of  a  core  logistics  function. 

The  purpose  of  this  research  was  to  examine  the  results  of  the  decision  to 
outsource  the  F-l  17  sustainment  tasks.  While  there  are  many  limiting  factors  to  our 
research,  such  as  the  limited  length  of  time  since  the  effort  was  outsourced,  the  almost 
immediate  deployment  to  Kosovo,  and  a  lack  of  competition  between  the  public  and 
private  sector,  many  conclusions  can  be  inferred  from  the  analysis. 
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The  chapter  ends  with  a  summary  of  conclusions  drawn  from  the  research, 
recommendations  on  how  this  research  might  be  employed  to  enhance  future  outsourcing 
decisions,  and  recommendations  for  future  studies. 

Review  of  Investigative  Questions 

Question  1 

Are  the  contract  performance  metrics  good  indicators  of  better  value? 

The  performance  metrics  selected  to  measure  contractor  performance  under  the 
TSPR  contract  are  identical  to  those  used  by  Air  Combat  Command  (ACC)  to  measure 
performance  of  all  aircraft  under  their  control.  The  metrics  used  to  measure  performance 
are:  mission  capable  rate,  total  non  mission  capable  rate,  MICAP  delivery,  RSP  kit  fill 
rate,  depot  delivery,  depot  quality,  delinquent  deficiency  report  rate,  and  weapon  system 
trainer  availability  rate.  We  had  two  criteria  for  measuring  the  suitability  of  the  selected 
performance  measurements.  The  first  criterion  is  whether  the  metrics  can  be  used  to 
compare  performance  before  the  TSPR  contract  with  performance  after  TSPR.  The 
second  criterion  is  whether  the  customer  perceives  the  metrics  as  adequate  performance 
indicators. 

We  used  historical  performance  data  for  the  pre-TSPR  FY97  and  FY98  periods 
and  actual  contractor  performance  data  accumulated  since  the  beginning  of  the  TSPR 
contract  on  1  October  1998  for  our  comparison.  We  found  the  metrics  adequate  to  use  as 
a  basis  of  comparison  of  performance  before  and  after  TSPR.  Results  from  the  customer 
satisfaction  questionnaire  were  used  to  determine  if  the  customers  perceived  the  metrics 
as  adequate  indicators  of  performance.  The  results  of  our  analysis  of  the  questionnaire 
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data  support  the  use  of  the  existing  performance  indicators.  In  addition,  the  expert 
interviews  also  supported  the  use  of  the  metrics,  but  suggested  that  intangibles  like 
working  relationship  and  responsiveness  should  be  considered. 

Question  2 

Has  the  decision  to  outsource  the  responsibility  for  the  F-117  depot-level  maintenance 
provided  a  better  value  to  the  Air  force? 

The  answer  to  this  question  forms  the  core  of  our  research.  We  collected  data 
from  three  sources  to  examine  this  question.  Historical  costs  for  F-117  sustainment  were 
gathered  from  the  F-l  17  System  Program  Office.  This  data  was  used  as  the  baseline  to 
compare  what  it  cost  to  perform  the  effort  before  the  TSPR  contract  was  awarded  to  the 
cost  of  performance  under  the  TSPR  contract.  Historical  cost  data  for  FY97  and  FY98, 
along  with  the  government’s  projection  of  “what  it  would  have  cost  the  government  to 
continue  the  same  level  of  support”  during  FY99  through  FY06  was  obtained.  A 
comparison  can  be  made  between  the  government’s  projected  costs  from  FY99  through 
FY06  to  the  actual  contractor  cost  under  TSPR.  Historical  technical  performance  data 
was  collected  for  the  FY97  and  FY98  time  periods.  This  data  was  compared  to 
performance  date  generated  under  the  TSPR  contract  to  determine  if  the  Air  Force  is 
obtaining  a  better  value  in  terms  of  performance. 

A  second  source  of  performance  data  was  elicited  from  the  customer  satisfaction 
questionnaires.  Responses  to  the  questionnaire  allowed  us  to  examine  the  customer’s 
perception  of  performance  before  TSPR  to  performance  after  TSPR.  We  can  draw 
conclusions  from  the  correlation  of  these  two  data  points  to  examine  if  the  Air  Force  is 
receiving  a  better  value  under  TSPR  from  the  customer’s  perspective. 
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Additionally,  we  solicited  comments  from  senior  level  experts  familiar  with 
F-l  17  sustainment  to  obtain  their  assessment  of  performance  before  and  after  TSPR.  To 
arrive  at  our  conclusion  we  triangulated  the  data  from  the  three  sources:  archival, 
customer  satisfaction  questionnaire,  and  expert  interviews. 

The  projected  savings  for  FY99,  based  on  the  contractor’s  estimate  to  complete, 
equate  to  $25.2  million.  Looking  solely  at  cost  as  the  indicator,  the  TSPR  contract  is 
providing  a  better  value  to  the  government. 

Our  analysis  indicates  that  there  is  little  difference  between  the  level  of 
sustainment  support  prior  to  TSPR  and  the  level  of  sustainment  support  after  award  of  the 
TSPR  contract.  The  final  driver,  in  the  case  of  the  F-l  17,  appears  to  be  cost. 

The  customer  satisfaction  questionnaire  indicates  that  the  customers  perceive 
sustainment  support  before  and  after  award  of  the  TSPR  contract  to  be  comparable. 
Question  3 

If  outsourcing  F-l  17  depot  maintenance  is  an  equal  or  better  value,  do  we  still  want  to 
give  up  the  potential  the  core  capability? 

Our  only  source  of  insight  into  this  question  is  the  expert  interviews.  One 
limiting  factor  affecting  our  research  is  the  short  length  of  time  since  the  TSPR  contract 
was  awarded.  Although  much  has  happened  during  that  period,  like  the  Kosovo 
deployment,  seven  months  of  performance  is  not  sufficient  time  from  which  to  draw 
conclusions. 

On  the  question  of  core  capability,  all  of  the  interviewees  stated  that  the 
government  has  lost  very  little  core  capability  due  to  the  fact  that  the  contractor  has 
always  performed  the  maintenance  on  this  aircraft.  However,  they  stated  that  we  did  lose 
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some  core  capability  in  sustaining  engineering  and  asset  management.  The  consensus 
was  that  the  loss  of  the  core  capability  was  small  when  compared  to  the  projected  $177.6 
million  financial  savings  over  the  life  of  the  TSPR  contract. 

Overall  Conclusion 

Although  our  findings  are  far  from  conclusive  evidence  that  the  Air  Force  as  a 
whole  is  receiving  a  better  value  by  outsourcing  their  depot  level  maintenance  functions. 
It  appears  we  are  saving  money  on  the  F-l  1 7  under  the  TSPR  contract.  It  is  important  to 
note  that  these  savings  are  based  on  the  available  cost  information  and  were  not  based  on 
the  governments  most  efficient  organization  analysis  as  required  by  A-76  regulations. 

Recommendations  for  Further  Use  of  This  Information 

With  the  elimination,  or  at  least  slow  down,  of  base  closures  as  the  primary  means 
of  reducing  costs  within  DoD,  outsourcing  has  become  the  method  of  choice  for  cost 
reduction  efforts.  The  results  of  this  analysis,  while  only  a  single  case  study,  indicate  that 
DoD  should  be  cautious  about  wholeheartedly  embracing  the  Commission  on  Roles  and 
Missions  (CORM)  and  Defense  Science  Boards  (DSB)  recommendations  to  outsource 
essentially  every  non-warfighting  activity.  There  is  potential  for  savings;  however,  there 
are  many  factors  to  consider.  DoD  needs  to  be  very  careful  about  making  outsourcing 
decisions  before  doing  the  research  (Saleck,  1998).  While  each  outsourcing  decision  is 
unique,  there  are  key  similarities.  There  are  costs  incurred  in  competitions  between  the 
public  and  private  sector.  There  are  costs  associated  with  eliminating  government 
civilian  jobs.  Many  times  the  personnel  savings  are  not  as  high  as  projected  since  many 
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displaced  civilian  employees  bump  into  lower  graded  jobs  while  retaining  their  current 
pay,  and  often  there  is  difficulty  in  comparing  performance  before  and  after.  All  of  these 
problems  can  be  overcome;  however,  these  issues  need  to  be  considered  before  making 
the  decision  to  outsource. 

Recommendations  for  Future  Research 

A  follow-on  study  should  be  accomplished  to  examine  the  decision  to  outsource 
F-l  17  sustainment  efforts  after  the  TSPR  contract  has  been  in  place  for  at  least  two  years. 
This  should  allow  sufficient  time  for  a  fair  comparison  of  performance  before  and  after 
the  TSPR  contract  was  awarded. 

A  study  should  be  accomplished  that  examines  the  overall  benefit  or  detriment  to 
the  DoD  from  outsourcing.  Once  the  decision  to  outsource  a  function  has  been  made,  the 
decision  is  nearly  irreversible;  infrastructure  is  eliminated  and  expertise  is  lost.  Rather 
than  look  at  a  single  circumstance,  the  overall  picture  needs  to  be  examined  to  determine 
if,  in  total,  outsourcing  is  detrimental  to  our  ability  to  sustain  a  prolonged  contingency. 

The  difficulty  in  making  an  accurate  comparison  lies  in  the  ability  of  the 
government  to  accurately  track  its  costs.  It  is  imperative  that  we  determine  our  true 
savings  before  projecting  these  savings  into  future  budgets.  A  joint  program  team 
consisting  of  contracting,  financial,  budget,  maintenance,  and  logistics  personnel  could 
focus  on  all  aspects  of  outsourcing  to  determine  true  cost  savings  and  the  overall  value  to 
the  Air  Force.  The  goal  would  be  to  have  each  member  provide  input  relating  to  their  job 
specialty.  Once  key  similarities  are  identified  a  model  can  be  built  that  can  be  applied  to 
any  outsourcing  situation  with  minimal  tailoring  for  the  specific  situation. 
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Appendix  A:  Definitions 

Key  terms  and  definitions  that  are  necessary  to  understand  the  major  aspects  of 
this  research  proposal  include: 

Outsourcing  -  is  the  use  of  federal  funds  to  pay  a  private  company  to  do  defense 
work  or  provide  a  service  for  a  defense  activity  (GAO  Report,  GAO/NSAID-97-86, 
1997). 


Privatization  -  is  the  complete  transfer  of  ownership  and  management  of  a 
function  to  the  private  sector,  but  DOD  pays  for  the  services  associated  with  the  function 
(GAO  Report,  GAO/NSAID-97-86, 1997). 

Better  Value  -  is  a  construct,  in  this  framework,  that  is  measured  by  whether  or 
not  there  are  actual  improvements  in  cost,  mission  capable  ratings  (MCR's),  quality, 
timeliness,  and  reliability  of  depot  repairs.  It  also  considers  whether  or  not  the 
knowledge,  expertise  and  experience  level  of  the  contractor's  depot-level  technicians  aids 
in  the  ability  to  answer  questions  and  fix  problems  more  rapidly  and  with  greater 
accuracy  than  with  the  organic  technicians.  Finally  it  includes  a  measure  of  a  perceived 
overall  improved  efficiency  of  the  working  relationship  between  flight  line  maintenance 
teams  and  the  depot. 

Modernization  -  is  a  key  goal  of  the  Air  Force  leadership,  outlined  in  Joint  Vision 
2001,  to  replace  or  upgrade  the  aging  Air  Force  weapon  systems.  This  goal,  due  to 
decreasing  budgets  and  manpower  resources,  is  what  drives  the  desire  to  save  money  by 
closing  bases  and  outsourcing  functions  that  are  not  inherently  governmental. 

Inherently  Governmental  Function  -  is  "a  function  that  is  so  intimately  related  to 
the  public  interest  as  to  mandate  performance  by  Government  employees?  These 
functions  include  those  activities  that  require  either  the  exercise  of  discretion  in  applying 
Government  authority  or  the  making  of  value  judgments  in  making  decisions  for  the 
Government"  (GAO  Report,  GAO/NSAID-97-86, 1997). 

The  following  definitions  come  directly  from  the  F-117  TSPR  Performance  Metrics 
(as  of  11  September  98). 

Total  Non  Mission  Capable  Supply  Rate  -  (TNMCS)  is  calculated  monthly  to 
establish  a  12-month  moving  average.  This  metric  is  based  upon  the  percentage  of 
aircraft  not  mission  capable  due  to  supply.  Non-Mission  Capable  Supply  (NMCS)  is 
additive  to  Non-Mission  Capable  Both  (NCMB)  to  compute  the  TNMCS. 

Scoring:  NMCS  rates  at  or  below  5.0%  will  receive  a  score  of  10.  From  5.1%  to  5.5%  = 
9,  from  5.6%  to  6.0%  =  8  etc.  All  fractions  will  be  rounded  up  to  the  nearest  tenth. 
NMCS  rates  of  9.6%  or  greater  it  will  receive  a  score  of  0. 

Premises: 

1 .  This  metric  pertains  to  LMSW-supplied  items  only  an  excludes  the  following: 
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NSN  items 

Local  Fabricated  Items  (SMR  code  MXX) 

Local  Assembled  Items  (SMR  code  AXX) 

Local  Purchased  Items 

Conditions: 

1 .  HAFB  will  calculate  the  supply  rate  (S-Rate)  monthly  and  provide  to  LMSW 
Support  Center  and  SPO  by  the  15th  of  each  month. 

2.  LMSW  can  request  relief  on  specific  items  due  to  maintenance-induced 
failures  of  low  stockage  items  (i.e.  damaged  noses,  wages,  etc.)  The 
contracting  officer  will  approve  or  deny  requests  for  relief. 

MICAP  Delivery  -  MICAP  response  time  will  be  calculated  monthly  to  establish  a 
twelve  month  moving  average.  This  metric  is  based  upon  the  time  taken  by  LMSW  to 
deliver  parts  on  MICAPs  to  HAFB.  A  MICAP  our  will  begin  upon  LMSW  notification, 
requirement  and  will  end  upon  receipt  of  the  item  in  HAFB  supply.  Logs  will  be  kept  at 
LMSW  and  HAFB  recording  the  time  and  date  of  the  MICAP  and  the  corresponding 
time/date  of  the  part  delivery.  Scoring:  Average  response  times  of  72  hours  or  less  will 
receive  a  score  of  10.  Average  response  times  greater  than  73  hours  and  up  to  84  hours 
will  receive  a  score  of  9.  Average  response  times  between  85  and  96  hours  will  receive  a 
score  of  8,  etc.  Average  response  times  greater  than  181  will  receive  a  score  of  0. 

Premises: 

1 .  This  metric  pertains  to  LMSW  supplied  items  only,  which  excludes  the 
following: 

NSN  items 

Local  Fabricated  Items  (SMR  code  MXX) 

Local  Assembled  Items  (SMR  code  AXX) 

Local  Purchased  Items 

2.  72  hours  applies  to  delivery  within  the  Continental  US  (CONUS)  only. 
Conditions: 

1 .  LMSW  can  request  relief  on  specific  items  I  created  disproportionate  negative 
impact  on  the  right.  The  contracting  officer  will  approve  or  deny  any  request 
for  relief. 

2.  LMSW  and  HAFB  logs  will  be  reconciled  and  reported  to  LMSW  Supply  and 
SPO  by  the  15th  of  each  month.  HAFB  will  have  the  final  decision  on  the 
report.  Monthly  report  will  be  submitted  to  LMSW  Supply  and  SPO. 

RSP  Kit  Fill  Rate  -  RSP  Kit  Fill  Rates  will  be  calculated  monthly  to  establish  a  12 
month  moving  average.  This  metric  is  based  upon  the  total  number  of  pieces  on  hand  in 
both  A  and  B  kits  divided  by  the  total  number  of  pieces  authorized.  This  rate  is 
expressed  as  a  percentage.  Scoring:  Fill  rates  at  97%  or  above  will  receive  a  score  of  10. 
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From  96%  to  96.9%  =  9, 95%  to  95.9%  =  8  etc.  All  fractions  will  be  rounded  up  to  the 
nearest  tenth.  Fill  rates  below  86%  will  receive  a  score  of  0. 

Premises: 

1 .  This  metric  pertains  to  LMSW-supplied  Recoverable  Items  (XD)  and 
excludes  the  following: 

NSN  items 

Local  Fabricated  Items  (SMR  code  MXX) 

Local  Assembled  Items  (SMR  code  AXX) 

Local  Purchased  Items 
Deployed  Kits 

2.  Kits  returning  from  deployment  shall  be  excluded  from  the  metric 
performance  measurement  for  30  days  to  allow  for  stock 
replenishment. 

Conditions: 

1 .  HAFB  will  compile  and  submit  a  report  to  LMSW  Supply  and  the 
SPO  by  the  15th  of  each  month. 

2.  Mission  Support  Kit  (MSK)  requirements  taken  out  of  RSP  still  need 
to  be  reported  as  part  of  RSP.  Deployment  of  one  RSP  Kit  will  not 
alter  the  metric  criteria.  Performance  will  continue  to  be  graded 
against  the  remaining  kit.  Should  all  RSP  Kits  be  deployed,  the  weight 
assigned  to  this  metric  will  be  assigned  to  NMCS  for  the  duration  of 
the  deployment.  When  the  kits  return,  the  12-month  moving  average 
will  be  reduced  by  the  months  the  kits  were  deployed. 

3 .  Should  one  or  both  kits  deploy  for  less  than  a  month,  the  kit  will  be 
considered  deployed  for  the  full  month. 

4.  Top  score  of  97%  applies  to  RSP  Kits  with  Low  Observable  (LO) 
considerable  materials.  Should  LO  be  removed  from  the  kits,  top 
score  becomes  96%.  The  contract  will  not  require  renegotiation  in  this 
instance. 

Depot  Quality  -  is  calculated  by  individual  aircraft  using  a  weighted  point  system 
that  is  based  upon  the  number  of  major  and  minor  discrepancies  found  after  the 
contractor  has  requested  DCMC  sign-off  for  closure.  After  each  inspection, 
representatives  from  LMSW  Quality  Assurance  and  DCMC  Quality  Assurance  will 
review  DCMC  write-ups.  Those  found  without  merit  will  be  documented  but  not 
considered  for  scoring.  DCMC  will  have  final  determination  of  merit. 
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Scoring:  Weighted  Point  System 

Major 

Minor 

Points 

Discrepancy  Count 

Points 

Discrepancy  Count 

0-4 

10 

0-10 

10 

5-9 

15 

11-20 

15 

10-14 

25 

21-30 

25 

15-up 

50 

31 -up 

50 

The  number  of  major  and  minor  discrepancies,  found  during  DCMC  Quality  Assurance 
Inspections,  on  each  aircraft  are  converted  to  points  per  aircraft.  Points  per  aircraft  will 
be  totaled  and  divided  by  the  number  of  aircraft  inspected  to  produce  a  12-month  moving 
average.  An  average  of  20  points  results  in  a  score  of  10;  21  to  25  is  a  score  of  9,  etc. 
Totals  of  65  points  or  higher  results  in  0  points. 

Premises: 

1 .  Major  Discrepancy  -  (Red  X  as  specified  in  TO  00-20-1):  the  weapon  system, 
supports  system,  or  equipment  is  considered  unsafe  or  unfit  for  flight  use. 

The  weapon  system,  support  system,  or  equipment  will  not  be  flown  or  used 
until  the  unsatisfactory  condition  is  corrected  or  symbol  cleared. 

2.  Minor  Discrepancy  —  is  an  unsatisfactory  condition,  but  the  condition  is  not 
sufficiently  urgent  or  dangerous  to  warrant  grounding  of  the  aircraft  or 
discontinuing  use  of  the  equipment. 

3.  A  list  of  Mandatory  Government  Inspections  (MGIs)  will  be  provided  to 
LMSW  Quality  Assurance  office.  The  MGI  list  may  change  due  to  the  work 
contracted. 

Conditions: 

1 .  DCMC  will  provide  a  quarterly  report  to  LMSW  Quality  Assurance  and  the 
SPO  by  the  15th  of  the  month  following  the  end  of  the  quarter. 

Depot  Delivery  -  The  depot  delivery  metric  is  calculated  for  each  aircraft  and  is 
based  upon  the  total  number  of  days  each  aircraft  fails  to  deliver  on  time.  Delivery  is 
tracked  by  aircraft  and  reflects  the  number  of  days  behind  schedule  an  aircraft  is  actually 
delivered.  The  number  of  days  is  totaled  after  each  delivery  and  then  averaged  across  the 
number  of  aircraft  delivered  to  establish  a  1 2-month  moving  average. 

Scoring:  Delivered  aircraft  will  be  scored  as  follows:  An  average  of  0  to  0.9  days  behind 
schedule  receives  a  score  of  10.  An  average  of  1 .0  to  1 .9  days  behind  schedule  receives  a 
score  of  8,  2.0  to  2.9  days  behind  schedule  receives  a  score  of  6,  3.0  to  3.9  days  behind 
schedule  receive  a  score  of  4,  4.0  to  4.9  days  behind  schedule  receives  a  score  of  2,  and 
aircraft  5  or  more  days  behind  schedule  receive  a  score  of  0. 

Premises: 

1 .  Total  days  behind  schedule  are  calculated  by  adding  the  number  of  total  days 
aircraft  were  not  delivered  on  the  scheduled  output  date. 


56 


2.  "On  time  delivery"  is  defined  as  all  depot  requirements  have  been  met  and 
wheels  are  up  and  locked  on  departure  from  the  depot. 

Conditions: 

1 .  Aircraft  will  be  delivered  in  accordance  with  the  negotiated  work  package. 
Changes  in  delivery  will  be  negotiated  between  LMSW,  DCMC  and  the  SPO. 

2.  DCMC  will  provide  a  quarterly  report  to  LMSW  Quality  Assurance  and  the 
SPO  by  the  15th  of  the  month,  following  the  end  of  the  quarter. 

Delinquent  Efficiency  Reports  (DRs)  -  Delinquency  tracking  is  performed 
monthly  to  establish  a  12-month  moving  average.  This  metric  is  based  on  the  average 
number  of  delinquent  DRs.  Scoring:  An  average  of  0  to  1  delinquent  DRs  for  the 
reporting  period  receives  a  score  of  10.  2  delinquent  DRs  receive  a  score  of  9, 3 
delinquent  DRs  receive  a  score  of  8,  etc.  1 1  or  more  delinquent  DRs  receive  a  score  of  0. 
Premises: 

1 .  DR  response  time  starts  upon  receipt  of  exhibit  at  LMSW  or  Supplier  Facility 
(if  shipped  direct). 

2.  Time  frames  for  this  metric  are  contained  in  TSPR  800,  dated  1  Oct.  98. 
Conditions: 

1 .  LMSW  will  provide  a  quarterly  report  to  the  SPO  by  the  1 5th  of  the  month, 
following  the  end  of  the  quarter. 

WST  Credited  Availability  -WST  Credited  Availability  is  calculated  monthly  to 
establish  a  12-month  moving  average.  Credited  availability  is  computed  as  follows: 

Ca  =  Tu  +  Td  -  Tc  +  Ta 
Ts  +  Ta 

Ca  =  Credited  Availability 

Tu  =  Utilization  Time  (Actual  WST  usage  for  the  period) 

Td  =  Total  Deviation  Time  (Events  such  as  the  weather,  power  outages,  etc.  that 
cannot  be  controlled  by  the  contractor). 

Tc  =  Chargeable  Maintenance  Deviations  (Unscheduled  maintenance  performed 
during  schedule  training  time). 

Ta  =  Alternate  Mission  Time  (Negative  if  time  lost)  (Time  spent  on  a  mission 
other  than  what  was  planned  for  training  because  of  degraded  WST 
performance). 

Ts  =  Scheduled  training  time 
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WST  Availability  metrics  by  fiscal  year  (FY)  shall  be  as  follows: 


FY 

Metric 

'99 

99% 

'00 

99% 

'01 

*93% 

'02 

*91% 

'03 

*89% 

'04 

*75% 

'05 

**97% 

'06 

**99% 

Scoring:  An  average  of  99%  or  greater  for  the  period  receives  a  score  of  10.  From  98.6 
to  98.9  receives  a  score  of  9,  etc.  An  average  of  95%  or  last  receives  a  score  of  0.  (In 
years  were  the  top  score  differs  from  99%  (FY'  01-05),  metric  scoring  will  begin  with  the 
number  shown  in  the  table  above  and  be  broken  down  accordingly.) 

Conditions: 

1 .  LMS  W  will  provide  a  quarterly  report  to  the  SPO  by  the  1 5  of  the  month, 
following  the  end  of  the  quarter. 

2.  *  -  The  AP500,  Array  Processor,  will  impact  of  the  WST  availability 
beginning  in  FY'  01 .  An  interim  solution  must  be  implemented  no  later  than 
FY01  or  the  Array  Processor  (AP500)  will  be  significantly  impacted  to  the 
point  where  it  would  become  nonfunctional.  If  this  occurs  or  when  the 
remaining  spares  are  no  longer  supportable  the  WST  availability  will  be 
calculated  on  all  functions  not  dependent  on  the  array  processor. 

3.  **  -  Upgrade  to  the  host  computer  suite  will  occur  in  FY'  04.  WST 
availability  will  reduce  each  fiscal  year  based  on  supportability  of  the  current 
host  computer  suite.  Should  upgrade  of  the  host  computer  suite  not  occur  in 
FY  04,  WST  availability  will  be  reduced  to  50%  in  FY'  05  and  0%  in  FY'  06. 

4.  Upgrade  to  the  host  computer  suite  is  considered  outside  the  scope  of  the 
TSPR  contract. 

General  Scoring/Performance  Calculations: 

"Total  Value"  consists  of: 

1.  Current  Performance 

2.  Score  -  Calculated  based  on  Current  Performance 

3 .  Weight  -  Normalizes  the  metric  to  the  weight  of  its  importance 

4.  Value  -  Score  x  Weight 

"Total  Value"  calculations  are  based  as  follows: 

1 .  Performance  values  are  translated  into  Scores  based  on  comparison  of  Current 
Performance  to  the  respective  scoring  guidelines  found  in  the  matrix. 
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2.  Wants  the  actual  Score  is  determined,  it  is  entered  as  the  Score  at  the  top  of 
the  chart. 

3.  That  Score  is  then  multiplied  by  the  Weight  assigned  to  the  respective  metric 
and  is  reflected  on  the  line  identified  as  Value. 

4.  Each  metric  has  been  assigned  a  metric  based  on  criticality  of  support  to 
HAFB. 

5.  Maximum  point  value  for  "Total  Value"  is  1,000  distributed  as  follows: 

-  CLIN  0001  700  Points 

-  NMCS 

-  MICAP 

-  RSP  Fill  Rate 

-  Delinquent  DRs 

-  WST  Availability 

-  CLIN  0002  300  Points 

-  DCMC  Depot  Quality  Assurance 

-  Depot  Delivery 

6.  Total  points  awarded  will  be  displayed  as  a  percentage  of  this  Total  Points 
Available.  This  percentage  will  be  the  Performance  Incentive  Fee  for  each 
CLIN. 

Example: 

Delinquent  DRs:  Current  Performance  =  0  Delinquent 
Score  =  10 
Weight  =  10 

Value  toward  Total  =  Score  (10)  x  Weight  (10)  =  Value  (100  Points) 
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Appendix  B:  Customer  Satisfaction  Questionnaire 


Demographics 


1. )  What  is  your  current  rank?  _ 

2. )  What  is  your  current  duty  title? _ _ _ 

3. )  How  many  years  experience  do  you  have  working  on  the  F  —117? 

0-1  2-3  4-5  6-7  8-9  9+ 

4. )  What  is  the  highest  education  level  you  have  completed? 

High  school  Associates  Degree  Bachelor’s  Degree  Masters  Degree  Ph.D. 


5.)  What  is  your  current  career  skill  level? 

3  level  5  level  7  level 

Performance  Questions 


Effective  1  October  1998,  a  contract  was  let  to  transfer  total  system  performance 
responsibility  for  depot  level  maintenance  to  a  contractor.  The  following  questions 
seek  to  measure  your  perceptions  of  the  difference  in  the  quality  of  support  you 
receive,  if  any,  comparing  support  for  the  periods  before  1  Oct  98  to  support  after 
lOct  98. 

6.)  The  number  of  aircraft  non  mission  capable  due  to  supply  is . 

1  2  3  4  5 

Much  Somewhat  No  Somewhat  Much 

Better  Better  Change  Worse  Worse 


7.)  The  time  it  takes  to  have  MICAP  parts  delivered  has 


1 

Greatly 

Decreased 


2 

Marginally 

Decreased 


3 

Has  Not 
Changed 


4 

Somewhat 

Increased 


5 

Greatly 

Increased 
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8.)  Replacement  Spare  Parts  (RSP)  Kit  fill  rates  are 


1  2  3  4  5 

Much  Somewhat  No  Somewhat  Much 

Better  Better  Change  Worse  Worse 


9. )  The  number  of  major  and  minor  discrepancies  found  after  depot  repair  has  .... 

1  2  3  4  5 

Greatly  Somewhat  Has  Not  Somewhat  Greatly 

Decreased  Decreased  Changed  Increased  Increased 

10. )  The  number  of  aircraft  being  delivered  behind  schedule  from  the  depot  has 

1  2  3  4  5 

Greatly  Somewhat  Has  Not  Somewhat  Greatly 

Decreased  Decreased  Changed  Increased  Increased 

11. )  The  number  of  Delinquent  Efficiency  Reports  (DR’s)  has . 

1  2  3  4  5 

Greatly  Somewhat  Has  Not  Somewhat  Greatly 

Decreased  Decreased  Changed  Increased  Increased 

12. )  The  overall  maintenance  relationship  between  the  depot  and  the  49th  is.... 

1  2  3  4  5 

Greatly  Somewhat  Has  Not  Somewhat  Much 

Improved  Improved  Changed  Worse  Worse 

13. )  Are  we  measuring  the  correct  performance  indicators?  Please  tell  us  what 

things  you  believe  should  be  measured  to  evaluate  the  contractors  performance 
in  regards  to  helping  you  perform  your  job  in  the  most  effective  and  efficient 
manner  possible. 
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